Getting Data In

How to configure parameter for TailingProcessor to automatically retry reading a file after failing?

NK_1
Path Finder

It took a while to copy the large logfile over the slow network, and it looks like the TailingProcessor gave up after 2 tries:


09-06-2014 15:17:03.607 -0700 WARN FileClassifierManager - Unable to open 'D:\Logs\syslog2014-09-05'.
09-06-2014 15:17:03.607 -0700 WARN FileClassifierManager - The file 'D:\Logs\syslog2014-09-05' is invalid. Reason: cannot_read
09-06-2014 15:17:03.607 -0700 INFO TailingProcessor - Ignoring file 'D:\Logs\syslog2014-09-05' due to: cannot_read
09-06-2014 15:22:47.707 -0700 WARN FileClassifierManager - Unable to open 'D:\Logs\syslog2014-09-05'.
09-06-2014 15:22:47.707 -0700 WARN FileClassifierManager - The file 'D:\Logs\syslog2014-09-05' is invalid. Reason: cannot_read
09-06-2014 15:22:47.707 -0700 INFO TailingProcessor - Ignoring file 'D:\Logs\syslog2014-09-05' due to: cannot_read

Any way to make it automatically retry a couple more times (via some configuration parameter somewhere)?
Using Splunk v6.1.3 for Windows.

0 Karma

lguinn2
Legend

Before ask Splunk to do this more times, have you checked:

  1. Does the file exist?
  2. Splunk is running under some user credentials. Is this file accessible by that user?
  3. Does another process have the file locked so that Splunk cannot read it?
0 Karma

lguinn2
Legend

Is there any way to set the copy process so that that it doesn't lock the file?

Is it possible to forward the file from its original location?

0 Karma

NK_1
Path Finder
  1. Yes.
  2. Yes.
  3. Yes, during the slow copy process.

If the file was small and/or the copy process was fast, Splunk indexes the file fine.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Splunkbase Unveils New App Listing Management Public Preview

Splunkbase Unveils New App Listing Management Public PreviewWe're thrilled to announce the public preview of ...

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...