Knowledge Management

Would building more than one summary index suit the needs of my issue?

prashanthberam
Explorer

Hi Everyone,
i am getting some events from application those events are about the hospital claims.
for every claim which have the info about req, ack, resp 3 messages. these 3 messages will have the unique id.
req and ack will get the messages within 1 minute . but the resp message will take 4 days sometimes.

in this case i am thinking to build the summary index for req and ack.
for rsp messages am thinking to build the one more summary index.
Anyone have an idea Please give me some suggestions thanks.

0 Karma

DalJeanis
Legend

You haven't told us what kind of issue you are having. There is nothing in what you have posted that indicates any reason to have a summary index at all.

The purpose of a summary index is to give an overview of aggregated data that saves you from having to look at the underlying data itself. The unique id, for a type of event where there will only ever be 3 events per unique id, is not something that I would expect to be in a summary index at all.

What are you thinking of summarizing, and over what timeframe?

0 Karma

prashanthberam
Explorer

i am thinking if i build the dashboard by using summary index. I will get the results much faster because per 15 minutes i will get more than 5000 claims. by using the summary index
I want to build an report like
how many transactions processed today
transactions count
response time between req and ack, ack- resp, timechart etc
if i want to display the report with req ack rsp correlationid with these fields.
here i have some confusion if today claim has processed i got the messages in for req and ack. after 2 days i got the rsp. if Choose the dashboard for the last 15 minutes, I will get the rsp message only.
i need some suggestions to build the dashboard with meaning fully.
any suggestions would be helpfull.

0 Karma

DalJeanis
Legend

Okay, given that situation, I would be considering the following statistics -

For req and ack, periodic volume. Just the number of each kind of items for that hour (or two hours, or four hours). if you want to get a little deeper, you can do stats for each period, on how long the delay was between the req and ack. (avg, min, max. stdev). Calculate this from the ack side, and dont' try to attach the stat to the req. (because the stats will be effectively identical. If you are calculating it hourly, then calculate it at the 15 minute mark, for the prior full hour. That gives any trailing transactions plenty of time to get indexed.

For response, if the typical delay is days, you don't need to produce summary stats very often. Every 6 hours or 8 hours or 12 hours or every day might be just fine. It doesn't HURT to do it more often, but it won't get you much extra information. (If you are going to be PULLING the current information frequently, though, then you might want to go ahead and run them on the same schedule.)

On the response side, you DEFINITELY want the stats on how long it took. Again, calculate those stats from the response side, since the event data is all in one place. In the last hour, we have received responses for requests from three weeks ago down to yesterday, with this average, this stdev, and so on.

Then, if you really want to, you can also calculate how long things took from the request side, but you need to wait until you have the vast majority of the responses. For example, calculate the stats on a rolling window where the requests happened exactly n days ago. (Where N is a fixed number between 8-14 )

You might learn some interesting things from that data, but from a statistical basis, across time, the request side stats are going to have an identical average and stdev to the response side stats. The difference will be in terms of finding out that, for example, requests early in the week get responses faster (in terms of calendar days) than requests late in the week.

Because of weekends, right?

0 Karma

somesoni2
Revered Legend

It will depends upon what type of summaries you want to created or what type of reporting you expect out of this data.

0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

Industry Solutions for Supply Chain and OT, Amazon Use Cases, Plus More New Articles ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...