Getting Data In

File system monitoring of text files that are overwritten

Michael_Schyma1
Contributor

The beginning and the end of the file are often the same, but we changed the data in the middle of the file, how do we get splunk to pick up the event and re-index it?

ALTERNITIVE: (Same Question)
monitoring of text files that are overwritten and timestamp changes but the files are often the same and-or have the same beginning, end and filename. We would like to force splunk to always re-index the file if it has a new timestamp. (df monitoring of files)

Tags (1)

bmacias84
Champion

If I understand your question correctly there are two setting that might help you accomplish this task.

crcSalt (cycling reducancy checks) which is mostly for log affects log rotation. This setting is located in the inputs.conf.

CHECK_METHOD which is the file check sum configuration. This setting has three configurations: endpoint_md5 (checksum of the first and last 256 bytes), entire_md5 (checksume the entire file), and modtime (checks only modification time of time). This can be found in the props.conf.

Additional Reading:

indexing 101

log file rotation

Hope this helps.

Channu
New Member

@bmacias84 I already have the crcSalt entry in my inputs.conf file.
Should I add the CHECK_METHOD = entire_md5 in the props.conf file? Will that help?

0 Karma
Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Dynamic formatting from XML events

This challenge was first posted on Slack #puzzles channelFor a previous puzzle, I needed a set of fixed-length ...

Enter the Agentic Era with Splunk AI Assistant for SPL 1.4

  🚀 Your data just got a serious AI upgrade — are you ready? Say hello to the Agentic Era with the ...

Stronger Security with Federated Search for S3, GCP SQL & Australian Threat ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...