Monitoring Splunk

Issue with UF not uploading files from a specific folder

Ismail_BSA
Path Finder

Hi,

I have an issue where my UF installed on a linux server is not uploading data to Splunk from a specific folder.

My inputs.conf file contains multiple simliar set ups for several folders to be uploaded. Everything is working perfectly except for one folder. In the inputs.conf file, I have the following set up for this folder:

[monitor:///DATA/remotelogs-ORACLE-MESS/test/*]
index=test_bd_oracle
sourcetype=test_oracle:audit:xml
host_segment = 3

This set up is to upload all files under path /DATA/remotelogs-ORACLE-MESS/test/ 

However, no files are being uploaded.

What  is also wierd is that when I open one of those files using the Linux Vim command, a temporary copy of that file is autocreated with extension .swp and the UF UPLOADS the .swp file.

Any help is appreciated.

Thank you.

Labels (1)
0 Karma
1 Solution

Ismail_BSA
Path Finder

By checking the  index="_internal" I found some errors related to my files that are not being uploded, here is an example

-0400 ERROR TailReader [3438 tailreader0] - File will not be read, is too small to match seekptr checksum (file=/file path/). Last time we saw this initcrc, filename was different. You may wish to use larger initCrcLen for this sourcetype, or a CRC salt on this source. Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.

I found this other question that seems simlar to my issue 

Solved: How do you fix this? ERROR TailingProcessor - File... - Splunk Community

and the resolution was to increase the initCrcLength value 

More details could be found on the official doc:  inputs.conf - Splunk Documentation

Thank you

 

View solution in original post

0 Karma

Ismail_BSA
Path Finder

By checking the  index="_internal" I found some errors related to my files that are not being uploded, here is an example

-0400 ERROR TailReader [3438 tailreader0] - File will not be read, is too small to match seekptr checksum (file=/file path/). Last time we saw this initcrc, filename was different. You may wish to use larger initCrcLen for this sourcetype, or a CRC salt on this source. Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.

I found this other question that seems simlar to my issue 

Solved: How do you fix this? ERROR TailingProcessor - File... - Splunk Community

and the resolution was to increase the initCrcLength value 

More details could be found on the official doc:  inputs.conf - Splunk Documentation

Thank you

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Ismail_BSA ,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Ismail_BSA,

the swp extension seems that the file wasn'r correctly closed. you have to manually delete the swp file to avoid the message, this could be the reason but I'm not sure.

Are you seure that there isn't any other copy of the file, eventually with a different name?

Splunk does't read a file twice unless you force double reading with the option "crcSalt = <SOURCE>" in inputs.conf.

Ciao.

Giuseppe

0 Karma

Ismail_BSA
Path Finder

Hi @gcusello 

The issue is that splunk is not Uploading my files but only uploading the .swp files (that are autocreated temporelly when I open the files with Vim command).

Are you seure that there isn't any other copy of the file, eventually with a different name?:

The files are recent, I created them my self and I am sure they are not duplicated.

I also tried to copy the entire folder /DATA/remotelogs-ORACLE-MESS/test/ to a new one /DATA/remotelogs-ORACLE-MESS/NEWtest/ and I created a new monitor Stanza but did not resolve my issue (no files are being uploded from the new folder)

 [monitor:///DATA/remotelogs-ORACLE-MESS/NEWtest/*]
index=test_bd_oracle
sourcetype=test_oracle:audit:xml
host_segment = 3

I also made sure to restart the Splunk Forwarder on my server.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Ismail_BSA,

as I said, manually delete the swp files.

The issue that copying the files in a new location they aren't read is related to the twice copy.

Ciao.

Giuseppe

0 Karma

Ismail_BSA
Path Finder

Hi @gcusello 

I already deleted the .swp files. The new files (.xml) are still not being uploaded

 

 

0 Karma

Ismail_BSA
Path Finder

Hi @scelikok 

Thank you for your reply.

However, I checked all monitor stanzas and they are not hitting the same path, I also renamed the original folder to "test" just to be sure it's unique and not interfearing with other paths.

 

0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @Ismail_BSA,

Your other monitor inputs may be hitting the same path, please check all other monitor stanzas if this happens.

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...