Getting Data In

TailReader Error then file uploaded a few hours later

katzr
Path Finder

I was trying to load data via my auto index and I was getting a tail reader error because I think Splunk was reading in my file as a duplicate (this was at ~2 pm). Then around 9 pm- the file uploaded fine with all of the data. I checked _internal and the only information I have that contain the source file name are the below messages:

-0400 INFO Metrics - group=per_source_thruput
-0400 INFO LicenseUsage - type=Usage

Does anyone know why file uploaded ~7 hours later and can help me troubleshoot?

Thank you!!

0 Karma

FrankVl
Ultra Champion

Can you at least share the exact TailReader error you observed initially?

What do the respective inputs.conf settings look like?

And how is that log file created / written to?

0 Karma

katzr
Path Finder

The errors I got when I first tried to upload were:

-0400 WARN FileClassifierManager - Unable to open
0400 WARN FileClassifierManager - The file is invalid. Reason: cannot_open
-0400 ERROR TailReader - error from read call

The inputs.conf looks like below:
whitelist = .csv$
disabled = false
index = it_snow_call_kiosk_logs_weekly
sourcetype = itcc:snow
initCrcLength = 640

And I was trying to upload a .csv.

0 Karma

FrankVl
Ultra Champion

How are you uploading that csv?

Sounds like splunk noticed the new file, but couldn't access it (because your upload process was keeping it fully locked somehow?)? And only later, it was able to access it and as a result still indexed it?

0 Karma

katzr
Path Finder

I was just dropping the .csv in the auto index folder- a simple copy and paste into the directory- do you know why a file could be locked? I appreciate your help!

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Event Series: Telemetry Pipeline Management

Balancing Scale and Spend: Gaining Control Over High-Volume Metrics in Splunk Observability Cloud As ...

Kick the Tires Before You Commit: A Hands-On Tour of the Splunk Observability Cloud ...

Evaluating an enterprise observability platform usually goes like this: fill out a form, get a free trial with ...

Deep insights, no barriers: Splunk Observability Cloud Free Edition

As software delivery cycles continue to accelerate, observability shouldn’t be a luxury — it should be a ...