Getting Data In

how to use crcSalt to index similar folders

mataharry
Communicator

I have a centralized server with all my logs per instance

 /var/log/database/hostA/report.log
 /var/log/database/hostA/report.log.1
 /var/log/database/hostA/report.log.2
 /var/log/database/hostB/report.log
 /var/log/database/hostB/report.log.1
 /var/log/database/hostB/report.log.2
 /var/log/database/hostC/report.log
 /var/log/database/hostC/report.log.1
 /var/log/database/hostC/report.log.2
 /var/log/database/hostD/report.log
 /var/log/database/hostD/report.log.1
 /var/log/database/hostD/report.log.2
etc ...

and a monitoring on each

[monitor://var/log/database/hostA/]
sourcetype=report
host=hostA

Except that the splunk monitor consider that all my reports.log are duplicates.B

10-06-2014 18:06:41.767 -0700 ERROR TailingProcessor - File will not be read, seekptr checksum did not match (file=/var/log/database/hostA/report.log). Last time we saw this initcrc, filename was different. You may wish to use a CRC salt on this source. Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.

If I want to use crcSalt=, it will cause all my rotated versions to be indexed.
Any better options ?

Tags (2)
1 Solution

yannK
Splunk Employee
Splunk Employee

You can use a static unique crcSalt for each monitor:

  • then all the files under the same stanza will use the same crcSalt and not be reindexed
  • while files in different stanza will have a different crc and be indexed.

    [monitor://var/log/database/hostA/]
    sourcetype=report
    host=hostA
    crcSalt=A
    [monitor://var/log/database/hostB/]
    sourcetype=report
    host=hostAB
    crcSalt=B

PS : It will not work with a generic input like

[monitor://var/log/database/*/]
 sourcetype=report
 hostsegment=4

View solution in original post

yannK
Splunk Employee
Splunk Employee

You can use a static unique crcSalt for each monitor:

  • then all the files under the same stanza will use the same crcSalt and not be reindexed
  • while files in different stanza will have a different crc and be indexed.

    [monitor://var/log/database/hostA/]
    sourcetype=report
    host=hostA
    crcSalt=A
    [monitor://var/log/database/hostB/]
    sourcetype=report
    host=hostAB
    crcSalt=B

PS : It will not work with a generic input like

[monitor://var/log/database/*/]
 sourcetype=report
 hostsegment=4
Get Updates on the Splunk Community!

Splunk at Cisco Live 2025: Learning, Innovation, and a Little Bit of Mr. Brightside

Pack your bags (and maybe your dancing shoes)—Cisco Live is heading to San Diego, June 8–12, 2025, and Splunk ...

Splunk App Dev Community Updates – What’s New and What’s Next

Welcome to your go-to roundup of everything happening in the Splunk App Dev Community! Whether you're building ...

The Latest Cisco Integrations With Splunk Platform!

Join us for an exciting tech talk where we’ll explore the latest integrations in Cisco + Splunk! We’ve ...