Splunk Search

How to pre-calculate a value per day and store it in a lookup to find averages in later searches?

Path Finder

Currently we have a search:

index="ecom" eventName | eventstats dc(sessionId) as totalnumberofsessions | search eventName = orderPlaced | stats dc(sessionId) as numbuyers max(totalnumberofsessions) as total | eval ConvRate = round(numbuyers*100/total,2) | table ConvRate

Essentially we're counting the # of people who order vs simply browse the site and create the #.

The problem is this search is taking longer and longer to do as traffic raises and/or the timeline we're looking at is longer.

We're looking to some how pre-compute the ConvRate per day and store that as a lookup some where in Splunk so that we can simply search the already calculated ConvRate for a series of days and average it.

I'm pretty sure Splunk can do this, but cannot recall how to do it. Can anyone give some pointers?

Any recommendations on better tags to represent this question also appreciated.

0 Karma
1 Solution

SplunkTrust
SplunkTrust

If you are going to do this calculation for longer period and which is dynamic, I would suggest to go with Summary indexing. Here is what you should do.

1) Create a scheduled search with following specification
SEARCH:

 index="ecom" eventName | eventstats dc(sessionId) as totalnumberofsessions | search eventName = orderPlaced | stats dc(sessionId) as numbuyers max(totalnumberofsessions) as total | eval ConvRate = round(numbuyers*100/total,2) | table ConvRate | addinfo | rename info_min_time as _time | table _time ConvRate 

Start time= -1d@d Finish time=@d (run query on yesterday's data)
Schedule:
Cron - 15 2 * * * (2:15 AM every day)

Summary Index = Enabled (provide appropriate index name that you want your daily summaries to be stored.

Once you set this up, you should receive one rows with yesterday's date and ConvRate for that day.

Now your can use following query (should be very fast) in your dashboards/reports.

index=YourSummaryIndex source="yourScheduledSearchName" |  stats avg(ConvRate) as AvgConvRate

See more information here : http://docs.splunk.com/Documentation/Splunk/6.2.5/Knowledge/Usesummaryindexing

View solution in original post

SplunkTrust
SplunkTrust

If you are going to do this calculation for longer period and which is dynamic, I would suggest to go with Summary indexing. Here is what you should do.

1) Create a scheduled search with following specification
SEARCH:

 index="ecom" eventName | eventstats dc(sessionId) as totalnumberofsessions | search eventName = orderPlaced | stats dc(sessionId) as numbuyers max(totalnumberofsessions) as total | eval ConvRate = round(numbuyers*100/total,2) | table ConvRate | addinfo | rename info_min_time as _time | table _time ConvRate 

Start time= -1d@d Finish time=@d (run query on yesterday's data)
Schedule:
Cron - 15 2 * * * (2:15 AM every day)

Summary Index = Enabled (provide appropriate index name that you want your daily summaries to be stored.

Once you set this up, you should receive one rows with yesterday's date and ConvRate for that day.

Now your can use following query (should be very fast) in your dashboards/reports.

index=YourSummaryIndex source="yourScheduledSearchName" |  stats avg(ConvRate) as AvgConvRate

See more information here : http://docs.splunk.com/Documentation/Splunk/6.2.5/Knowledge/Usesummaryindexing

View solution in original post

Path Finder

I actually already tried this and while the report runs it dones't seem to save it to the index. Do I need to create a new index from scratch to get it working?
EDIT - I was looking for results from today it made it already for yesterday. Thanks!

0 Karma