Knowledge Management

Summary index approach with aggregate data

Path Finder


I would like to summarize some data with aggregated statistic results. When I summarize a search like

(search statement...)    | sitimechart span=5m count(sasl_username)  sum(nrcpt)  avg(nrcpt)  by sasl_username

I can find the aggregate data in psrsvd fields. But, as I can read in there isn't a field for avg, so in my search the "avg(nrcpt)" is ignored.

Also, the use of psrsvd fields is unsupported. So, with summarized data I can do a search like

index=summary |  rename psrsvd_sm_nrcpt as sumDEST | table _time sumDEST | stats sum(sumDEST)

It works, but it is unsupported.

Another approach is to use collect. I tried in many way, finally my above summarizing search has the same result with

| sort _time
| streamstats time_window=5m count(sasl_username) AS NumMSG sum(nrcpt) AS NUMDST avg(nrcpt) AS MeanDST by sasl_username
| sort -_time
| bin _time span=5m
| dedup sasl_username _time
| sort _time
| table _time sasl_username NumMSG NUMDST MeanDST
| collect

...and now I preserve also the MeanDST field. But this search seems to be more complex than the equivalent with sitimechart.

What is the best approach?

Many thanks

0 Karma

Path Finder

Uhm, I now understand this doc

where I read "If you use these commands you can use the same search string that you use for the search that you eventually run on the summary index, with the exception that you use regular reporting commands in the latter search".

So I should ignore at all the summary reserved fields: for a kind of magic I just retype the same command during the indexing phase without the "si" prefix and my result appears!
It works, and also the avg metrics which doesn't have a reserved field reports the expected results!

But... let suppose I would like a complex search like this:

      | sort 0 _time
      | streamstats time_window=1h sum(psrsvd_sm_nrcpt) as sumXDest by sasl_username
      | timechart span=1h count(sasl_username) as "N. Msg 1h" sum(nrcpt) as "N. Dest. 1h" avg(sumXDest) as "Avg N Dest last 1h per account"  avg(nrcpt) as "Avg 5mdest last 1h"

I see no values for avg(sumXdest). To have this to work, I have to eval or rename the psrsvd_sm_nrcpt field, and put it in a table. Again, I have to switch to the previous approach.

I think to have understood power but also limits in summary index... If I want more flexibility I have to switch to unsupported approach. I'll try to avoid this.

Thank you

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...