Security

On the capture of structured data

oda
Communicator

By default Splunk assumes the same file when the first 256 bytes are the same.

How is Splunk structured data judged?
For example csv file.

props.conf


[testcsv]
INDEXED_EXTRACTIONS=csv


If the file names are the same, it seems to be regarded as the same file.

Do you know something?
Is there a description in the manual?

For files with the same name, the second time is not recognized as a header line.

Please give me some help.
Thanks.

Tags (1)
0 Karma

adonio
Ultra Champion

hello there,
check this link for crcSalt:
https://docs.splunk.com/Documentation/Splunk/7.1.3/Admin/Inputsconf

 crcSalt = <string>
    * Use this setting to force the input to consume files that have matching CRCs
      (cyclic redundancy checks).
        * (The input only performs CRC checks against, by default, the first 256
          bytes of a file. This behavior prevents the input from indexing the same
          file twice, even though you may have renamed it -- as, for example, with
          rolling log files. However, because the CRC is based on only the first
          few lines of the file, it is possible for legitimately different files
          to have matching CRCs, particularly if they have identical headers.)
    * If set, <string> is added to the CRC.
    * If set to the literal string <SOURCE> (including the angle brackets), the
      full directory path to the source file is added to the CRC. This ensures
      that each file being monitored has a unique CRC.   When crcSalt is invoked,
      it is usually set to <SOURCE>.
    * Be cautious about using this setting with rolling log files; it could lead
      to the log file being re-indexed after it has rolled.
    * In many situations, initCrcLength can be used to achieve the same goals.
    * Defaults to empty.

more to read here:
http://docs.splunk.com/Documentation/Splunk/7.1.3/Data/Howlogfilerotationishandled

hope it helps

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...