Issue with UF not uploading files from a specific folder

[Ismail_BSA]

Hi,

I have an issue where my UF installed on a linux server is not uploading data to Splunk from a specific folder.

My inputs.conf file contains multiple simliar set ups for several folders to be uploaded. Everything is working perfectly except for one folder. In the inputs.conf file, I have the following set up for this folder:

[monitor:///DATA/remotelogs-ORACLE-MESS/test/*]
index=test_bd_oracle
sourcetype=test_oracle:audit:xml
host_segment = 3

This set up is to upload all files under path /DATA/remotelogs-ORACLE-MESS/test/ 

However, no files are being uploaded.

What  is also wierd is that when I open one of those files using the Linux Vim command, a temporary copy of that file is autocreated with extension .swp and the UF UPLOADS the .swp file.

Any help is appreciated.

Thank you.

[Ismail_BSA · Accepted solution]

By checking the  index="_internal" I found some errors related to my files that are not being uploded, here is an example

-0400 ERROR TailReader [3438 tailreader0] - File will not be read, is too small to match seekptr checksum (file=/file path/). Last time we saw this initcrc, filename was different. You may wish to use larger initCrcLen for this sourcetype, or a CRC salt on this source. Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.

I found this other question that seems simlar to my issue 

Solved: How do you fix this? ERROR TailingProcessor - File... - Splunk Community

and the resolution was to increase the initCrcLength value 

More details could be found on the official doc:  inputs.conf - Splunk Documentation

Thank you

[gcusello]

Hi @Ismail_BSA ,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉

[gcusello]

Hi @Ismail_BSA,

the swp extension seems that the file wasn'r correctly closed. you have to manually delete the swp file to avoid the message, this could be the reason but I'm not sure.

Are you seure that there isn't any other copy of the file, eventually with a different name?

Splunk does't read a file twice unless you force double reading with the option "crcSalt = <SOURCE>" in inputs.conf.

Ciao.

Giuseppe

[Ismail_BSA]

Hi @gcusello 

The issue is that splunk is not Uploading my files but only uploading the .swp files (that are autocreated temporelly when I open the files with Vim command).

Are you seure that there isn't any other copy of the file, eventually with a different name?:

The files are recent, I created them my self and I am sure they are not duplicated.

I also tried to copy the entire folder /DATA/remotelogs-ORACLE-MESS/test/ to a new one /DATA/remotelogs-ORACLE-MESS/NEWtest/ and I created a new monitor Stanza but did not resolve my issue (no files are being uploded from the new folder)

 [monitor:///DATA/remotelogs-ORACLE-MESS/NEWtest/*]
index=test_bd_oracle
sourcetype=test_oracle:audit:xml
host_segment = 3

I also made sure to restart the Splunk Forwarder on my server.

[gcusello]

Hi @Ismail_BSA,

as I said, manually delete the swp files.

The issue that copying the files in a new location they aren't read is related to the twice copy.

Ciao.

Giuseppe

[Ismail_BSA]

Hi @gcusello 

I already deleted the .swp files. The new files (.xml) are still not being uploaded

[Ismail_BSA]

Hi @scelikok 

Thank you for your reply.

However, I checked all monitor stanzas and they are not hitting the same path, I also renamed the original folder to "test" just to be sure it's unique and not interfearing with other paths.

[scelikok]

Hi @Ismail_BSA,

Your other monitor inputs may be hitting the same path, please check all other monitor stanzas if this happens.