Monitoring Splunk

Does Disk Block Size matter? Performance increase?

bmacias84
Champion

This has probably been asked before but I will do it again.

A classic debate over the years in RDBs is of block size: smaller block size for transactional systems and larger for data ware house systems. I know that Splunk not a RDB and that this is an OS configuration, but would performance be increased by decreasing or increase the Disk Block Size? By default I believe NTFS and ext2/3 are 4KB. Has anyone done any performance tests?

Only doc I could find even related to the the topic: PerformanceTroubleshooting

1 Solution

gkanapathy
Splunk Employee
Splunk Employee

The same considerations apply for Splunk as for relational databases. It depends how you're planning to use it. If you are performing rare searches, a slightly smaller block size may improve performance, because you're not transferring as much data. If you're performing dense searches and reports, you may read fewer blocks, though this effect is probably not large as the block reads are likely to be sequential.

Either way though, with filesystem block sizes in the normal range of 4kb to 64kb and with modern disk sizes and disk transfer times, this is unlikely to make a measurable difference.

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

The same considerations apply for Splunk as for relational databases. It depends how you're planning to use it. If you are performing rare searches, a slightly smaller block size may improve performance, because you're not transferring as much data. If you're performing dense searches and reports, you may read fewer blocks, though this effect is probably not large as the block reads are likely to be sequential.

Either way though, with filesystem block sizes in the normal range of 4kb to 64kb and with modern disk sizes and disk transfer times, this is unlikely to make a measurable difference.

bmacias84
Champion

Valid response, but has anyone or Splunk performed any testing regarding this?

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Unlocking Unified Insights: New Gigamon Federated Search App for Splunk

In today’s data-heavy environment, organizations are caught in a data distribution dilemma. As data volumes ...

GA: New Data Management App in Splunk Platform

Streamlining Data Management: Introducing a unified experience in Splunk Managing data at scale shouldn’t feel ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...