Hello @ajitshukla61116 ,
I know a partial solution for your question - use initCrcLength = 1048576, splunk will calculate CRC sum based on the first 1MB and reindex the whole csv file if anything changed in the first 1048576 bytes of the file.
[monitor:///tmp/file.csv]
disabled = false
....
initCrcLength = 1048576
here is an excerpt from the documentation:
initCrcLength = <integer>
* How much of a file, in bytes, that the input reads before trying to
identify whether it is a file that has already been seen. You might want to
adjust this if you have many files with common headers (comment headers,
long CSV headers, etc) and recurring filenames.
* Cannot be less than 256 or more than 1048576.
* CAUTION: Improper use of this setting causes data to be re-indexed. You
might want to consult with Splunk Support before adjusting this value - the
default is fine for most installations.
* Default: 256 (bytes).
... View more