topic Re: Forwarder only reads file if i save it, ignores when script saves it in Getting Data In

Forwarder only reads file if i save it, ignores when script saves it

LAcioffi — Mon, 04 May 2020 13:31:52 GMT

Hello everyone,

I have Splunk Universal Forwarder running on a server watching a few files for changes. Log data is inserted at the end of the files every 5 minutes or so.
Up until a few days ago, all files were working and being correctly monitored. Today i noticed that a single file out of 10+ is not being monitored correctly.

When the script appends something to the file and closes it (thus updating the update date), the data doesn't arrive at the index. However, if i open the file, change anything and save it, all the data that should have arrived suddenly arrives.

This problem started out of the blue. I tried restarting the universal forwarder service, changing how the file is saved, deleting the file and letting the script re-create it, everything, but it still won't work.

Any ideas? Has this ever happened to anyone before?

P.S.: The file is open and closed explictly in my script. All other files do the same thing and work, only this one file is giving me trouble.

Thanks!

Re: Forwarder only reads file if i save it, ignores when script saves it

PavelP — Mon, 04 May 2020 13:49:39 GMT

Hello @LAcioffi ,

this can be a problem with CRC calculation, so splunk doesn't notice that the file has been changed. This especially the case if the file has large header or/and is too small.

Run this command on the UF and compare the output after the file has been changed by the script:

./splunk _internal call /services/admin/inputstatus/TailingProcessor:FileStatus

if you see that this command reports wrong status and size, then you need to change crcSalt or initCrcLength in your inputs.conf

Please let me know if it worked for you

Re: Forwarder only reads file if i save it, ignores when script saves it

LAcioffi — Mon, 04 May 2020 14:19:21 GMT

Hey,

I don't have permissions to do that, but i checked the forwarder logs and the admin checked for error logs on index=_internal and we didn't find anything.

Re: Forwarder only reads file if i save it, ignores when script saves it

PavelP — Mon, 04 May 2020 14:55:17 GMT

Hello @LAcioffi

there are no errors in such situations.

Can you post a redacted file?