Why is summary index search returning duplicate and incorrect counts of data?

[smaran06]

Hi Team,

I am populating the data in summary index using the following Splunk search

index=data"  | sistats count as total by  appName,trueclient, httpstatus,request_uri

when, I do stats over this

index=summary_ |stats count as total by appName.

We are getting lot of difference in counts.

When, I run the search directly the app counts are very low, then on summary index it's very high. Why summary index data is returning wrong data? is it because, I added trueclient, httpstatus,request_uri in sistats?

[briancronrath]

When you search for just

index=summary_

How many different source values are you getting? Are you getting sources outside of the search you used to populate it? If so, limit down to just the name of the search as your source when you search and see if the numbers look better.

[somesoni2]

When you use the si* command for summary index, you need to use the same aggregation command on the summary index data. Give this a try and see if the counts are matching.

index=summary | stats count as total by  appName,trueclient, httpstatus,request_uri | stats sum(total) as total by appName

compare with this.

index=data |stats count as total by appName

[smaran06]

Thanks, still the count is not matching, summary index is at very high when compare to data which is not is summary index

[graa1005]

I have exactly the same problem. Multiple entries in the summary index for the same data. Only one value for info_search_time So it looks like it is one search. only outputs on multiple indexers.
If i deleted the summary data and re runtje job to add the results to the summary index.I get double data only different as the previous summary data. So only a part of the data is double.

[graa1005]

Some data can exist multiple times. IN my case up to 5 times. (i have 5 indexers)

[ddrillic]

We normally put a safe-guard to avoid duplicates. A left join in the spirit of - | join type=left <field> [search index=<summary index name>| eval matched="Y"]

[graa1005]

ddrillic my search does NOT generate duplicates. I execute the search ones a day to generate a summary of the records of the day before.

[ddrillic]

Sorry ; -)