Splunk Search

crcsalt query

Gowtham0809
New Member

I have some CSV files indexed via splunk. I have noticed that files are getting indexed daily even though there is no changes made in the file. for example I have a file which is indexed on 27th July for the first time. after that there is no changes made in the file so far. but the file is being indexed everyday the same date. Is this behaviors is due to crcsalt=, This crc command is defined in the inputs .conf file for monitoring the data?

Thanks

Tags (1)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi Gowtham0809,
crcSalt is an option useful when you want to reindex a file already indexed, that usually Splunk doesn't index twice.

crcSalt = <string>
* Use this setting to force the input to consume files that have matching CRCs
  (cyclic redundancy checks).
    * By default, the input only performs CRC checks against the first 256
      bytes of a file. This behavior prevents the input from indexing the same
      file twice, even though you might have renamed it, as with rolling log
      files, for example. Because the CRC is based on only the first
      few lines of the file, it is possible for legitimately different files
      to have matching CRCs, particularly if they have identical headers.
* If set, <string> is added to the CRC.
* If set to the literal string "<SOURCE>" (including the angle brackets), the
  full directory path to the source file is added to the CRC. This ensures
  that each file being monitored has a unique CRC. When 'crcSalt' is invoked,
  it is usually set to <SOURCE>.
* Be cautious about using this setting with rolling log files; it could lead
  to the log file being re-indexed after it has rolled.
* In many situations, 'initCrcLength' can be used to achieve the same goals.
* Default: empty string.
* List item

So you don't need of it in your case.

Anyway, also with crcSalt, your file shouldn't be indexed twice until it maintain the same name.

Bye.
Giuseppe

View solution in original post

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi Gowtham0809,
crcSalt is an option useful when you want to reindex a file already indexed, that usually Splunk doesn't index twice.

crcSalt = <string>
* Use this setting to force the input to consume files that have matching CRCs
  (cyclic redundancy checks).
    * By default, the input only performs CRC checks against the first 256
      bytes of a file. This behavior prevents the input from indexing the same
      file twice, even though you might have renamed it, as with rolling log
      files, for example. Because the CRC is based on only the first
      few lines of the file, it is possible for legitimately different files
      to have matching CRCs, particularly if they have identical headers.
* If set, <string> is added to the CRC.
* If set to the literal string "<SOURCE>" (including the angle brackets), the
  full directory path to the source file is added to the CRC. This ensures
  that each file being monitored has a unique CRC. When 'crcSalt' is invoked,
  it is usually set to <SOURCE>.
* Be cautious about using this setting with rolling log files; it could lead
  to the log file being re-indexed after it has rolled.
* In many situations, 'initCrcLength' can be used to achieve the same goals.
* Default: empty string.
* List item

So you don't need of it in your case.

Anyway, also with crcSalt, your file shouldn't be indexed twice until it maintain the same name.

Bye.
Giuseppe

0 Karma

Gowtham0809
New Member

Hello Giuseppe,

I have question, Here I have a situation, I have some 100 entries in file named demo_21_08_19. which already indexed in to splunk. Now the very next day I am creating a file with the same 100 entries but in the name of demo_22-07_19. also I have configured CRCSALT= in inputs.conf. Now I question is will the entries from the second file demo_22-07_19 though the data is already got indexed in demo_21_08_19 the previous day.

I want all the data should get indexed from new file, even its is already got indexed.

Thanks

0 Karma

gcusello
SplunkTrust
SplunkTrust

Sorry but I don't understand your question.
Anyway, if you use crcSalt = <SOURCE>, you restart forwarder and you modify the filename, the content of the new file is indexed again.

Bye.
Giuseppe

0 Karma

Gowtham0809
New Member

Hello,

I will be creating new files on the daily basis in the directory which is been monitored, But the files will not be of same name. they have the date printed in the name. I want the whole file to be indexed even there is no new data in the file when compared to previous day.

Hope I made it clear now

Thanks

0 Karma

gcusello
SplunkTrust
SplunkTrust

If you have every day a file with a different name and you use the option crcSalt = <SOURCE> , the content of the new file will be sent to Indexers for indexing, also if the content is the same of the previous day.

Bye.
Giuseppe

0 Karma

Gowtham0809
New Member

Thanks for the clarification

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...