Splunk Dev

.gz file not getting indexed in splunk

beenagulzar
New Member

i am getting this error , every time when i am indexing the .csv.gz file
updated less than 10000ms ago, will not read it until it stops changing.
has stopped changing , will read it now .

inputs.conf :
[monitor:///tmp/*.csv.gz]
sourcetype=test

props.conf:
[test]
CHECK_FOR_HEADER = true
DATETIME_CONFIG =
INDEXED_EXTRACTIONS = csv
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
disabled = false
REPORT-AutoHeader = AutoHeader-6
category = Structured

Tags (1)
0 Karma

dbcase
Motivator

The line

crcSalt=/tmp/.csv.gz

is incorrect

please use (verbatium)

crcSalt= <SOURCE>
0 Karma

beenagulzar
New Member

i have added the crcsalt file like below

[monitor:///tmp/.csv.gz]
sourcetype=test
crcSalt=/tmp/
.csv.gz
ignoreOlderThan=1d

but still i cant see todays sample1.csv.gz file

in log file i can find out

Handling file =/tmp/sample1.csv.gz
ArchivedProcessor - reading Path = /tmp/sample1.csv.gz ( seek=0 len=142048)

but not seeing the data in the splunk indexer.

its showing handling file , reading file but not seeign finished processing file . Kindly need your input .

0 Karma

dbcase
Motivator

try crcSalt= <SOURCE> in your inputs.conf, also you may want to use batch instead of monitor with a move_policy = sinkhole so it will erase the previous file when indexed.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

That's not an error, Splunk is informing you that it's not going to read the archive until it's confident that the archive has stopped changing.

0 Karma

beenagulzar
New Member

No , my env is , everyday new file will be added in that location to monitor , e.g. /tmp/sample1.csv.gz ... , sample2.csv.gz ....only first time it went through ... from day 2 always it's throwing the same info but no data in the indexer . My sample1.csv.gz has the first line in common like same fields everyday .. but from the second line it's different ... is it because of that ... u can find my props.conf and my inputs.conf in my first post .

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

So... when you run this over all time, you see nothing? index=main sourcetype=test

Anything here? index=_internal group=per_source_thruput series=*csv.gz*

0 Karma

beenagulzar
New Member

No , just showing read it now ... but I am not seeing any data in the indexer , from this particular source.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Take Action Automatically on Splunk Alerts with Red Hat Ansible Automation Platform

 Are you ready to revolutionize your IT operations? As digital transformation accelerates, the demand for ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...

Financial services organizations face an impossible equation: maintain 99.9% uptime for mission-critical ...