https://community.splunk.com/t5/Monitoring-Splunk/What-are-the-performance-impact-or-other-ramifications-of/m-p/84946#M1047 <P>We are trying out a method of deleting events from the index. It looks like a metadb rebuild takes place every time we do this so would like to know what the performance implications are, if any:</P> <PRE><CODE>10-09-2012 16:01:13.287 +0000 INFO databasePartitionPolicy - Detected a deletion, rebuilding metadata in directory '/var/lib/splunkdb/main' 10-09-2012 16:01:38.986 +0000 INFO databasePartitionPolicy - rebuildMetadata called: full=true path=/var/lib/splunkdb/main reason= delete_fixup 10-09-2012 16:02:05.529 +0000 INFO databasePartitionPolicy - Detected a deletion, rebuilding metadata in directory '/var/lib/splunkdb/main' 10-09-2012 16:02:30.585 +0000 INFO databasePartitionPolicy - rebuildMetadata called: full=true path=/var/lib/splunkdb/main reason= delete_fixup 10-09-2012 16:02:56.595 +0000 INFO databasePartitionPolicy - Detected a deletion, rebuilding metadata in directory '/var/lib/splunkdb/main' 10-09-2012 16:03:23.114 +0000 INFO databasePartitionPolicy - rebuildMetadata called: full=true path=/var/lib/splunkdb/main reason= delete_fixup 10-09-2012 16:03:48.600 +0000 INFO databasePartitionPolicy - Detected a deletion, rebuilding metadata in directory '/var/lib/splunkdb/main' 10-09-2012 16:04:14.583 +0000 INFO databasePartitionPolicy - rebuildMetadata called: full=true path=/var/lib/splunkdb/main reason= delete_fixup 10-09-2012 16:04:40.996 +0000 INFO databasePartitionPolicy - Detected a deletion, rebuilding metadata in directory '/var/lib/splunkdb/main' 10-09-2012 16:05:06.330 +0000 INFO databasePartitionPolicy - rebuildMetadata called: full=true path=/var/lib/splunkdb/main reason= delete_fixup 10-09-2012 16:05:32.331 +0000 INFO databasePartitionPolicy - Detected a deletion, rebuilding metadata in directory '/var/lib/splunkdb/main' </CODE></PRE> Tue, 09 Oct 2012 17:44:11 GMT the_wolverine 2012-10-09T17:44:11Z https://community.splunk.com/t5/Monitoring-Splunk/What-are-the-performance-impact-or-other-ramifications-of/m-p/84946#M1047 <P>We are trying out a method of deleting events from the index. It looks like a metadb rebuild takes place every time we do this so would like to know what the performance implications are, if any:</P> <PRE><CODE>10-09-2012 16:01:13.287 +0000 INFO databasePartitionPolicy - Detected a deletion, rebuilding metadata in directory '/var/lib/splunkdb/main' 10-09-2012 16:01:38.986 +0000 INFO databasePartitionPolicy - rebuildMetadata called: full=true path=/var/lib/splunkdb/main reason= delete_fixup 10-09-2012 16:02:05.529 +0000 INFO databasePartitionPolicy - Detected a deletion, rebuilding metadata in directory '/var/lib/splunkdb/main' 10-09-2012 16:02:30.585 +0000 INFO databasePartitionPolicy - rebuildMetadata called: full=true path=/var/lib/splunkdb/main reason= delete_fixup 10-09-2012 16:02:56.595 +0000 INFO databasePartitionPolicy - Detected a deletion, rebuilding metadata in directory '/var/lib/splunkdb/main' 10-09-2012 16:03:23.114 +0000 INFO databasePartitionPolicy - rebuildMetadata called: full=true path=/var/lib/splunkdb/main reason= delete_fixup 10-09-2012 16:03:48.600 +0000 INFO databasePartitionPolicy - Detected a deletion, rebuilding metadata in directory '/var/lib/splunkdb/main' 10-09-2012 16:04:14.583 +0000 INFO databasePartitionPolicy - rebuildMetadata called: full=true path=/var/lib/splunkdb/main reason= delete_fixup 10-09-2012 16:04:40.996 +0000 INFO databasePartitionPolicy - Detected a deletion, rebuilding metadata in directory '/var/lib/splunkdb/main' 10-09-2012 16:05:06.330 +0000 INFO databasePartitionPolicy - rebuildMetadata called: full=true path=/var/lib/splunkdb/main reason= delete_fixup 10-09-2012 16:05:32.331 +0000 INFO databasePartitionPolicy - Detected a deletion, rebuilding metadata in directory '/var/lib/splunkdb/main' </CODE></PRE> Tue, 09 Oct 2012 17:44:11 GMT https://community.splunk.com/t5/Monitoring-Splunk/What-are-the-performance-impact-or-other-ramifications-of/m-p/84946#M1047 the_wolverine 2012-10-09T17:44:11Z https://community.splunk.com/t5/Monitoring-Splunk/What-are-the-performance-impact-or-other-ramifications-of/m-p/84947#M1048 <P>From experience, it appears that DELETE is relatively expensive. When performing DELETE against many events (more than 10k), if done repeatedly in succession, for example, in the case of scheduled search that finds and "DELETE"s, bad things can happen if your system is undersized so watch it carefully. No idea what happens in a properly sized implementation but maybe Splunk can answer.</P> Wed, 17 Oct 2012 06:29:25 GMT https://community.splunk.com/t5/Monitoring-Splunk/What-are-the-performance-impact-or-other-ramifications-of/m-p/84947#M1048 the_wolverine 2012-10-17T06:29:25Z