Hello,
I have a problem that changes in the monitored CSV files are not detected and indexed.
The file name is not changed, but the file is always overwritten and updated.
The header of the CSV file has 207 bytes, so I've add initCrcLength = 512 on inputs.conf.
I guess it doesn't make sense since the file name doesn't change, but I also add crcSalt = <SOURCE>.
Any help would be appreciated.
Note: In my environment, I have built a Splunk server in AWS and monitor CSV files stored in EBS.
[ inputs.conf ]
[monitor:///tmp/upload/targetFolder]
disabled = false
host_regex = \\(.*)\/
index = targetData
sourcetype = targetData
crcSalt = <SOURCE>
initCrcLength = 512
[ props.conf ]
CHARSET = UTF-8
BREAK_ONLY_BEFORE_DATE =
DATETIME_CONFIG =
HEADER_FIELD_LINE_NUMBER = 1
INDEXED_EXTRACTIONS = csv
KV_MODE = none
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIMESTAMP_FIELDS = SpecifiedColumn
TIME_FORMAT = %Y/%m/%d
category = Structureds
description = Something
disabled = false
pulldown_type = true
Thank you.
When I encounter this problem I'll typically set initCrCLength to something long like 4096.
Sometimes, when really barely anything in a file changes yet I still want it ingested when its modified each day, i'll add a CHECK_METHOD = modtime to props.conf
Hope that helps.