Splunk Search

Why is summary index search returning duplicate and incorrect counts of data?

smaran06
Path Finder

Hi Team,

I am populating the data in summary index using the following Splunk search

index=data"  | sistats count as total by  appName,trueclient, httpstatus,request_uri

when, I do stats over this

index=summary_ |stats count as total by appName.

We are getting lot of difference in counts.

When, I run the search directly the app counts are very low, then on summary index it's very high. Why summary index data is returning wrong data? is it because, I added trueclient, httpstatus,request_uri in sistats?

0 Karma

briancronrath
Contributor

When you search for just

index=summary_

How many different source values are you getting? Are you getting sources outside of the search you used to populate it? If so, limit down to just the name of the search as your source when you search and see if the numbers look better.

0 Karma

somesoni2
Revered Legend

When you use the si* command for summary index, you need to use the same aggregation command on the summary index data. Give this a try and see if the counts are matching.

index=summary | stats count as total by  appName,trueclient, httpstatus,request_uri | stats sum(total) as total by appName

compare with this.

index=data |stats count as total by appName
0 Karma

smaran06
Path Finder

Thanks, still the count is not matching, summary index is at very high when compare to data which is not is summary index

0 Karma

graa1005
Explorer

I have exactly the same problem. Multiple entries in the summary index for the same data. Only one value for info_search_time So it looks like it is one search. only outputs on multiple indexers.
If i deleted the summary data and re runtje job to add the results to the summary index.I get double data only different as the previous summary data. So only a part of the data is double.

0 Karma

graa1005
Explorer

Some data can exist multiple times. IN my case up to 5 times. (i have 5 indexers)

0 Karma

ddrillic
Ultra Champion

We normally put a safe-guard to avoid duplicates. A left join in the spirit of - | join type=left <field> [search index=<summary index name>| eval matched="Y"]

0 Karma

graa1005
Explorer

ddrillic my search does NOT generate duplicates. I execute the search ones a day to generate a summary of the records of the day before.

0 Karma

ddrillic
Ultra Champion

Sorry ; -)

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!