Monitoring Splunk

Deletion of event data in a index for performance

linu1988
Champion

Hello,
I would like to know if deletion of events which are not required will increase the search performance? They are in very big numbers which slowed my search down on the dashboard.

If not do i have to clean the existing index or do we have some other solution?

Thanks

Tags (3)
0 Karma

mloven_splunk
Splunk Employee
Splunk Employee

Deletion of data (via the 'delete' command) won't increase performance. It's sort of a misnomer. The 'delete' command won't actually delete any data from your indexes, it will only make the data 'invisible' to searches.

Cleaning out an index is certainly an option, but a drastic one. If you don't mind losing ALL data from your index, you can go that route.

I'd start looking at the underlying causes of WHY your searches are slow.

Are you piping everything into one index? Maybe look at separating your data into different indexes. This should make searches (prepended with index=) run a bit faster.

Over what time range are you running your searches? If you're constantly running searches "over all time", then you should get out of that habit. Only run a search over the time range you need.

How many scheduled saved searches do you have running? If you're running Splunk on an underpowered server, your ad-hoc search may be contending with scheduled saved searches (or other users running ad-hoc searches) for CPU cycles.

aelliott
Motivator

alternatively you could set "expiration" times, or expiration per amount of data, by default data is stored for 6 years.
http://answers.splunk.com/answers/4236/how-to-deleteoverwrite-data-older-than-x-number-of-days

http://docs.splunk.com/Documentation/Splunk/6.0/Indexer/Setaretirementandarchivingpolicy

0 Karma

linu1988
Champion

The platform/the dashboard configuration isn't a problem. I wouldn't be so happy to reset the index by which I would loose my required data.

I can't separate the index as all were the same set of data of similar log. However due to some test logs million of records are now present in the index, which is the cause of performance that I understand. When showing the specific source of data of a given category now it's taking very long hence I was thinking of deleting the records. Thank you for your suggestion!!!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...