Monitoring Splunk

Does Disk Block Size matter? Performance increase?

bmacias84
Champion

This has probably been asked before but I will do it again.

A classic debate over the years in RDBs is of block size: smaller block size for transactional systems and larger for data ware house systems. I know that Splunk not a RDB and that this is an OS configuration, but would performance be increased by decreasing or increase the Disk Block Size? By default I believe NTFS and ext2/3 are 4KB. Has anyone done any performance tests?

Only doc I could find even related to the the topic: PerformanceTroubleshooting

1 Solution

gkanapathy
Splunk Employee
Splunk Employee

The same considerations apply for Splunk as for relational databases. It depends how you're planning to use it. If you are performing rare searches, a slightly smaller block size may improve performance, because you're not transferring as much data. If you're performing dense searches and reports, you may read fewer blocks, though this effect is probably not large as the block reads are likely to be sequential.

Either way though, with filesystem block sizes in the normal range of 4kb to 64kb and with modern disk sizes and disk transfer times, this is unlikely to make a measurable difference.

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

The same considerations apply for Splunk as for relational databases. It depends how you're planning to use it. If you are performing rare searches, a slightly smaller block size may improve performance, because you're not transferring as much data. If you're performing dense searches and reports, you may read fewer blocks, though this effect is probably not large as the block reads are likely to be sequential.

Either way though, with filesystem block sizes in the normal range of 4kb to 64kb and with modern disk sizes and disk transfer times, this is unlikely to make a measurable difference.

bmacias84
Champion

Valid response, but has anyone or Splunk performed any testing regarding this?

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...

Laser Bananas and Edge Hubs: Exploring Operational Technology (OT) Data Through a ...

  OT is a different environment to traditional IT and can have interesting challenges when interfacing the ...

Event Series: Mastering AI Tokenomics and Splunk Agent Observability

Beyond the Black Box: Correlating AI Performance and Tokenomics with Splunk Agent Observability   As ...