Hi All, have this dilemma where source counts does not match the count inserted in summary index. sample query that was used - Base search: index=sample_index
| rex mode=sed field=author"s/(\w|\d|[\D\W])/*/g"
| eval raw_event=_raw
| rex mode=sed field=raw_event"s/(:?author\=[\w|\d|\D\W]+)/author= *********/g"
| fields user owner ip mac_address input_file dest_file log_name orig_time orig_sourcetype act category default message message_id raw_mac severity tag vendor product then summary indexing is enabled. runs every 30 minutes that gets past 30 minutes. Validation: if base search is used, for example, it has a result of 100k events. when checked in summary index it has only 50% or less inserted. Note that not all fields are present in all events. example for owner field, it has 3 Values, 17.377% of events. Question: does it summarize the fields being inserted to the summary index, where it drops the fields with null values? or is that the expected behavior from summary indexing? Thanks!
... View more