Monitoring Splunk

Do not detect changes in the monitored CSV file.

JA8eginner
Explorer

Hello,

I have a problem that changes in the monitored CSV files are not detected and indexed.
The file name is not changed, but the file is always overwritten and updated.
The header of the CSV file has 207 bytes, so I've add initCrcLength = 512 on inputs.conf.
I guess it doesn't make sense since the file name doesn't change, but I also add crcSalt = <SOURCE>.
Any help would be appreciated.

Note: In my environment, I have built a Splunk server in AWS and monitor CSV files stored in EBS.

[ inputs.conf ]

 

[monitor:///tmp/upload/targetFolder]
disabled = false
host_regex = \\(.*)\/
index = targetData
sourcetype = targetData
crcSalt = <SOURCE>
initCrcLength = 512

 


[ props.conf ]

 

CHARSET = UTF-8
BREAK_ONLY_BEFORE_DATE =
DATETIME_CONFIG =
HEADER_FIELD_LINE_NUMBER = 1
INDEXED_EXTRACTIONS = csv
KV_MODE = none
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIMESTAMP_FIELDS = SpecifiedColumn
TIME_FORMAT = %Y/%m/%d
category = Structureds
description = Something
disabled = false
pulldown_type = true

 


Thank you.

Labels (1)
0 Karma

splunk219783
Path Finder

When I encounter this problem I'll typically set initCrCLength to something long like 4096.

Sometimes, when really barely anything in a file changes yet I still want it ingested when its modified each day, i'll add a CHECK_METHOD = modtime to props.conf

Hope that helps.

 

 

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!