topic How to make sure that the data forwarded is loading in the searchhead/indexer completely? in Getting Data In

How to make sure that the data forwarded is loading in the searchhead/indexer completely?

manjunathmeti — Tue, 29 Sep 2020 09:00:57 GMT

I have a forwarder installed on a server and I am extracting the data for indexes like Name,Class etc and while extracting I am also storing extraction information as shown below in extraction_audit index. This index also forwarded to indexer along with indexes Name, Class,Place so on.

index...............begin_extraction_time......end_extraction_time......total_records_extracted
Name ...............1457452861....................... ..145745287..........................150
Class................1457453260..........................1457453290........................100
School_details..1457453564..........................1457453575.........................25
Place................1457453612 .........................1457453620 ........................50
Other................1457453734..........................1457453744.........................10
Name................1457453892..........................1457453895.........................25
Place................1457454094..........................1457454096..........................5

Now how can I check in search heads count by index is same as total_records_extracted between begin_extraction_time and end_extraction_time.

For Example for index=Name between 1457452861 and 145745287, how to check count by index is equal or not equal to total_records_extracted which is in index=extraction_audit.

Re: How to make sure that the data forwarded is loading in the searchhead/indexer completely?

jkat54 — Tue, 08 Mar 2016 13:12:41 GMT

Hello,

This should work:

index=name _index_earliest=1457452861 _index_latest=145745287

Thanks,
Michael

Re: How to make sure that the data forwarded is loading in the searchhead/indexer completely?

jkat54 — Tue, 08 Mar 2016 13:14:38 GMT

if you want to do it all in one search...

index=name _index_earliest=1457452861 _index_latest=145745287 | stats c AS count1 | appendcols [search index=extraction_audit  _index_earliest=1457452861 _index_latest=145745287 | stats c AS count2] | eval matching=if(count1=count2,"True","False") | table count1, count2, matching | rename count1 as "Indexed Event Count", count2 as "Audited Event Count"