topic Re: Multiple logs being written per day; only the first log is searchable in Getting Data In

Multiple logs being written per day; only the first log is searchable

bspalding — Thu, 12 Jun 2025 17:46:54 GMT

I have an application writing multiple log files per day - the files are very similar to each other. The file naming convention is logfile_MM-DD-YYYY_hh-mm.log (e.g. logfile_06-12-2025-11-47.log).

My universal forwarder is set up like this:

[monitor://E:\path\logfile*.log]
disabled = 0
crcSalt = <SOURCE>
index = XXXX
sourcetype = XXXX
_meta = env::prod-new

The first log file of the day is searchable in Splunk, but every file after that is not visible.

I have tried using logfile_*.log as the file name. I have also tried without the crcSalt command, but I'm not seeing any difference.

Any suggestions?

Re: Multiple logs being written per day; only the first log is searchable

richgalloway — Thu, 12 Jun 2025 23:47:52 GMT

Do the files have a common header? If so, you may need to set initCrcLength to a value larger than the header.

Re: Multiple logs being written per day; only the first log is searchable

PrewinThomas — Fri, 13 Jun 2025 03:45:46 GMT

@bspalding

Use initCrcLength if your files are extremely similar at the start and the UF is getting confused

Eg:
Note-Change initCrcLength value based on your similar header size

[monitor://E:\path\logfile*.log]
disabled = 0
initCrcLength = 256
crcSalt = <UNIQUESOURCE>
index = XXXX
sourcetype = XXXX
_meta = env::prod-new

Regards,
Prewin
Splunk Enthusiast | Always happy to help! If this answer helped you, please consider marking it as the solution or giving a kudos/Karma. Thanks!