Monitoring Splunk

Do not detect changes in the monitored CSV file.

JA8eginner
Explorer

Hello,

I have a problem that changes in the monitored CSV files are not detected and indexed.
The file name is not changed, but the file is always overwritten and updated.
The header of the CSV file has 207 bytes, so I've add initCrcLength = 512 on inputs.conf.
I guess it doesn't make sense since the file name doesn't change, but I also add crcSalt = <SOURCE>.
Any help would be appreciated.

Note: In my environment, I have built a Splunk server in AWS and monitor CSV files stored in EBS.

[ inputs.conf ]

 

[monitor:///tmp/upload/targetFolder]
disabled = false
host_regex = \\(.*)\/
index = targetData
sourcetype = targetData
crcSalt = <SOURCE>
initCrcLength = 512

 


[ props.conf ]

 

CHARSET = UTF-8
BREAK_ONLY_BEFORE_DATE =
DATETIME_CONFIG =
HEADER_FIELD_LINE_NUMBER = 1
INDEXED_EXTRACTIONS = csv
KV_MODE = none
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIMESTAMP_FIELDS = SpecifiedColumn
TIME_FORMAT = %Y/%m/%d
category = Structureds
description = Something
disabled = false
pulldown_type = true

 


Thank you.

Labels (1)
0 Karma

splunk219783
Path Finder

When I encounter this problem I'll typically set initCrCLength to something long like 4096.

Sometimes, when really barely anything in a file changes yet I still want it ingested when its modified each day, i'll add a CHECK_METHOD = modtime to props.conf

Hope that helps.

 

 

0 Karma
Get Updates on the Splunk Community!

Splunk Security Content for Threat Detection & Response, Q1 Roundup

Join Principal Threat Researcher, Michael Haag, as he walks through:An introduction to the Splunk Threat ...

Splunk Life | Happy Pride Month!

Happy Pride Month, Splunk Community! &#x1f308; In the United States, as well as many countries around the ...

SplunkTrust | Where Are They Now - Michael Uschmann

The Background Five years ago, Splunk published several videos showcasing members of the SplunkTrust to share ...