Deployment Architecture

Archiving speed is very very slow

sudhir7
Explorer

I tried to archive data by adding frozenTimePeriodInSecs and coldToFrozenDir settings for individual indexes in local/indexes.conf.
We have 10TB of data to be archive but the rate of archiving was very very slow around 1GB/day.
Has anyone else faced a similar situation ?

0 Karma

Elsurion
Communicator

Your whole index is 10TB and a the day to freeze is how many GB?
My system here is freezing about 10GB/day, but only because the events should be frozen. The Index is about 1TB.
Also the discs are not highspeed just a bunch of normal discs.

0 Karma

sudhir7
Explorer

Hi Elsurion,
Thanks for your reply.
We are using SATA disk, could that be the reason for slowness ?

0 Karma

Elsurion
Communicator

I don't think so. Unless you have just one disc which is used for others as well and not a whole cabinet.
But you could run bonnie++ to check your filesystem for benchmark issues.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...