Deployment Architecture

rotating log files with large headers.

kkalmbach
Path Finder

I have a log file that gets rotated (foo.log becomes foo.log.1, foo.log.1 becomes foo.log.2, etc).
Every time the file gets rotated, the new foo.log gets a common header (which is more that 256 bytes long)

Splunk believes that this file has already been indexed (since the first 256 bytes are identical).
I can't add "crcSalt = " since the source filename is always the same.

Are there any other ways to get this file read every time it gets rotated?

Thanks
-Kevin

Tags (1)
0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

I don't know of a good way around this problem. It comes up fairly often and is something Splunk should be able to fix, either to use a larger CRC, or to offset some amount before the CRC. Please file an Enhancement Request with Splunk to help us get this fixed.

View solution in original post

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

I don't know of a good way around this problem. It comes up fairly often and is something Splunk should be able to fix, either to use a larger CRC, or to offset some amount before the CRC. Please file an Enhancement Request with Splunk to help us get this fixed.

0 Karma
Get Updates on the Splunk Community!

Data Management Digest – December 2025

Welcome to the December edition of Data Management Digest! As we continue our journey of data innovation, the ...

Index This | What is broken 80% of the time by February?

December 2025 Edition   Hayyy Splunk Education Enthusiasts and the Eternally Curious!    We’re back with this ...

Unlock Faster Time-to-Value on Edge and Ingest Processor with New SPL2 Pipeline ...

Hello Splunk Community,   We're thrilled to share an exciting update that will help you manage your data more ...