Getting Data In

Estimating size of blockSignature index

dwaddle
SplunkTrust
SplunkTrust

Is there any data available on estimating how big an indexer's blockSignature database may become, based the blockSignSize and daily event volumes?

Tags (2)

jrodman
Splunk Employee
Splunk Employee

blockSIgnature events are just listings of eventIDs. Therefore a block signature event is just a bunch of numbers like 45:35 with spaces in between. Therefore the size of the block signature database has no correspondence with the size of the original events, making this difficult to estimate in bytes.

However, each event requires a price of around 8 bytes in the text, precompressed, and each event requires a hash and indexing-related overhead. I would guesstimate that each block, in in the current rawdata format and tsidx format, is going to cost on the order of a kilobyte. Empirical data is obviously vastly superior to my guesses.

There's therefore the obvious tunable here which is the blockSignSize. I think anyone using this feature significantly probably wants to tune that value upwards. You don't want it too high for low incoming datarates, but at higher rates, plausibly values like 10000 might be reasonable.

0 Karma

Michael_Wilde
Splunk Employee
Splunk Employee

Good question. Mine is the same size of my main index. Not thrilled.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Cisco Data Fabric from Architecture to Investigation, Better SOC Visibility, and More ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

The Trust Gap: Why a Data Foundation is Fundamental to an Agentic Enterprise

The Trust Gap: Why a data foundation is fundamental to an  Agentic Enterprise.   Agentic AI is transforming ...

Data Management Digest – September 2026

    Welcome to the September 2026 edition of Data Management Digest! September brought a fresh wave of ...