Monitoring Splunk

Do not detect changes in the monitored CSV file.

JA8eginner
Explorer

Hello,

I have a problem that changes in the monitored CSV files are not detected and indexed.
The file name is not changed, but the file is always overwritten and updated.
The header of the CSV file has 207 bytes, so I've add initCrcLength = 512 on inputs.conf.
I guess it doesn't make sense since the file name doesn't change, but I also add crcSalt = <SOURCE>.
Any help would be appreciated.

Note: In my environment, I have built a Splunk server in AWS and monitor CSV files stored in EBS.

[ inputs.conf ]

 

[monitor:///tmp/upload/targetFolder]
disabled = false
host_regex = \\(.*)\/
index = targetData
sourcetype = targetData
crcSalt = <SOURCE>
initCrcLength = 512

 


[ props.conf ]

 

CHARSET = UTF-8
BREAK_ONLY_BEFORE_DATE =
DATETIME_CONFIG =
HEADER_FIELD_LINE_NUMBER = 1
INDEXED_EXTRACTIONS = csv
KV_MODE = none
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIMESTAMP_FIELDS = SpecifiedColumn
TIME_FORMAT = %Y/%m/%d
category = Structureds
description = Something
disabled = false
pulldown_type = true

 


Thank you.

0 Karma

splunk219783
Path Finder

When I encounter this problem I'll typically set initCrCLength to something long like 4096.

Sometimes, when really barely anything in a file changes yet I still want it ingested when its modified each day, i'll add a CHECK_METHOD = modtime to props.conf

Hope that helps.

 

 

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Data Persistence in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. What happens if the OpenTelemetry collector ...

Introducing Splunk 10.0: Smarter, Faster, and More Powerful Than Ever

Now On Demand Whether you're managing complex deployments or looking to future-proof your data ...

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...