topic Re: Summary Index sistats not working with xyseries in Knowledge Management

Summary Index sistats not working with xyseries

ravimishrabglr — Sat, 06 Jun 2020 17:14:40 GMT

I have Two Questions:

1st Questions: Below is the query to generate stats that I want to push into Summary Index:

index="myIndex" host="myHost" source="/var/logs/events.log" sourcetype="ss:vv:events" (MTHD="POST" OR MTHD="GET")
| rex field=U "(?P[^\/]+)(\/([a-z0-9]{32})|$)"
| search (ApiName=abc OR ApiName=xyz)
| dedup CR,RE
| stats count as TotalReq by ApiName, Status
| xyseries ApiName Status, TotalReq
| addtotals labelfield=ApiName col=t label="ColTotals" fieldname="RowTotals"

It gives me perfect result as:

ApiName | 200 | 400 | 404 | 500 | RowTotals
abc | 12 | 2 | 4 | 1 | 19
xyz | 10 | 3 | 2 | 2 | 17
ColTotals | 22 | 5 | 6 | 3 | 36

But when I am changing stats to sistats to push into Summary Index, it is not producing any result, please help me with the query.

2nd Question: I already have a Summary Index available and one stats report with different query is already been pushed everyday, which I have annotated using Add Fields option in Edit Summary Index window as report = firstReport, now can I push another (above) report into same Summary Index with different annotation as report = secondReport? will it work or I have to create another Summary Index for this report also, Please help.

Re: Summary Index sistats not working with xyseries

to4kawa — Thu, 05 Mar 2020 00:54:47 GMT

Re: Summary Index sistats not working with xyseries

ravimishrabglr — Fri, 06 Mar 2020 08:30:33 GMT

@to4kawa 1st worked, able to push data in summary index. but while fetching it is not aggregating result. it is returning 2 time same api name and counts in 2 days search. running collect query daily in a scheduled report.

Re: Summary Index sistats not working with xyseries

to4kawa — Wed, 30 Sep 2020 04:30:28 GMT

2nd answer |eval report="secondReport" is to next search.

next search example:
`index=your_summary report="secondReport" you_want

Re: Summary Index sistats not working with xyseries

ravimishrabglr — Thu, 19 Mar 2020 17:19:29 GMT

Thanks @to4kawa It worked but I ran into another strange problem, here is my query-

index="myIndex" host="myHost" source="/var/logs/events.log" sourcetype="ss:vv:events" (MTHD="POST" OR MTHD="GET")
| rex field=U "(?P[^\/]+)(\/([a-zA-Z0-9]{32})|$)"
| where ApiName IN ("abc", "xyz")
| dedup CR,RE
| stats count as TotalReq by ApiName, Status
| xyseries ApiName Status, TotalReq

Now when I am using IN command in above query like this:
| where ApiName IN ("abc") produce 10 events and
| where ApiName IN ("xyz") produce 20 events but when using both ApiName together
| where ApiName IN ("abc", "xyz") produce 25 events only, it should produce 30 right? where are 5 events?

Please help me how to fix this problem. Plus CCing wonderful people who already helped me in other question: @richgalloway @gf13579 @vnravikumar

Re: Summary Index sistats not working with xyseries

ololdach — Thu, 19 Mar 2020 17:36:36 GMT

Hi,

sistats creates the summary index and doesn't output anything. It is an alternative to the collect suggested above. To report from the summaries, you need to use a stats. I have copied this from the documentation of the sistats command:

Create a summary index with the statistics about the average, for each hour, of any unique field that ends with the string "lay". For example, delay, xdelay, relay, etc.

 ... | sistats avg(*lay) BY date_hour

To create a report, run a search against the summary index using this search

index=summary | stats avg(*lay) BY date_hour

Hope it helps
Oliver

Re: Summary Index sistats not working with xyseries

ravimishrabglr — Fri, 20 Mar 2020 05:39:40 GMT

@ololdach Thanks for the answer, but i know that. my problem was when i was using sistats in above query, it was producing no data to push into summary index, but it should produce data with additional info columns prefixed with psrsvd, when you run query with sistats it gives output and that output is kept in summary index so that we can query later.

Re: Summary Index sistats not working with xyseries

to4kawa — Fri, 20 Mar 2020 10:40:25 GMT

index="myIndex" host="myHost" source="/var/logs/events.log" sourcetype="ss:vv:events" (MTHD="POST" OR MTHD="GET")
| rex field=U "(?P<ApiName>[^\/]+)(\/([a-zA-Z0-9]{32})|$)"
| stats count by ApiName

Check count and provide them here

Re: Summary Index sistats not working with xyseries

ravimishrabglr — Fri, 20 Mar 2020 11:34:22 GMT

In above, i provided count (10, 20) just for example, but below are real the count from old query and the new query that you provided.
ApiName | oldQuery | newQuery
abc | 8258875 | 21781751
xyz | 74371 | 2283504

Re: Summary Index sistats not working with xyseries

to4kawa — Fri, 20 Mar 2020 21:01:33 GMT

I don't know your oldQuery and newQuery.

your question is Summary Index sistats not working with xyseries
Do you resolve this?

If you have other issues, you should ask with another question

Re: Summary Index sistats not working with xyseries

woodcock — Sat, 21 Mar 2020 20:43:10 GMT

I see little reason to use sistats most of the time because prestats formatted data is difficult to read and near-impossible to debug; therefore I have never used it. Instead, I always use stats. This just means that when you leverage the summary index data, you have to know what you are doing and do it correctly, which is the case with normal events and SPL so you already have the skills that you need. So I suggest that you do it according to this document:
https://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Configuresummaryindexes#Manually_configure_a_report_to_populate_a_summary_index

As far as using multiple populating searches to dump into a single summary index, you should make that decision the same way that you would deciding whether to put multiple sourcetypes into a regular events index. Just make sure that you use a different report_name value (which is the equivalent of sourcetype when doing summary index because all data in a summary index has a sourcetype value of stash).