Getting Data In
Highlighted

Monitor log files with spaces in the file name

Contributor

hi,

I am having issues with splunk universal forwarder monitoring log files with spaces in the name . The file is a regular text file and not binary , but the forwarder is considering the file as binary

Example of log file name :

"NMDox_PRD.EP6XWBSDE26931 Started 2017-09-14.txt"

12-20-2017 22:55:47.122 -0800 WARN FileClassifierManager - The file '/esclogs/OLD/NMDoxPRD.EP6XWBSDE26931 Started 2017-09-14.txt' is invalid. Reason: binary
12-20-2017 22:55:47.123 -0800 INFO TailReader - Ignoring file '/esclogs/NMDoxPRD.EP6XWBSDE26931 Started 2017-09-14.txt' due to: binary

inputs.conf
[monitor:///edslogs/]
disabled = false
whitelist = .txt$
index = esd
prod
sourcetype = esd:trace
host_regex = (EP\d\w+)
crcSalt =

Appreciate any guidance on this problem.
Thanks

0 Karma
Highlighted

Re: Monitor log files with spaces in the file name

SplunkTrust
SplunkTrust

Hi @nmohammed,

This problem might occur when there will be garbage character in your txt file. Can you please check file type using command file NMDox_PRD.EP6XWBSDE26931 Started 2017-09-14.txt and let us know the output.

0 Karma
Highlighted

Re: Monitor log files with spaces in the file name

Contributor

hi @harsmarvania57

file NMDox_PRD.EP6XWBSDE26931\ Started\ 2017-09-14.txt

NMDox_PRD.EP6XWBSDE26931 Started 2017-09-14.txt: Little-endian UTF-16 Unicode English text, with very long lines, with CRLF line terminators

0 Karma
Highlighted

Re: Monitor log files with spaces in the file name

SplunkTrust
SplunkTrust

Looks like binary, can you please try to read log files using less command less NMDox_PRD.EP6XWBSDE26931 Started 2017-09-14.txt ? If it's binary then it will ask you that file is in binary continue anyway ? After that give yes and try to find those special/garbage character in that file.

0 Karma
Highlighted

Re: Monitor log files with spaces in the file name

Contributor

Those log files are written by an .NET application running on Windows onto a CIFS share. I have mounted the CIFS share on a linux server, I had issues of extreme slowness and lag monitoring logs directly from CIFS shares directly using a universal forwarder running on Windows server.

Now there 100's of such logs files that need to be monitored and written continuously.

0 Karma
Highlighted

Re: Monitor log files with spaces in the file name

SplunkTrust
SplunkTrust

So you are not facing slowness issues on Linux server ? As mentioned by @Elsurion, you can try to set CHARSET or if you want to read binary file anyway then you can set NO_BINARY_CHECK = true in props.conf

NO_BINARY_CHECK = [true|false]
* When set to true, Splunk processes binary files.
* Can only be used on the basis of [<sourcetype>], or [source::<source>],
  not [host::<host>].
* Defaults to false (binary files are ignored).
* This setting applies at input time, when data is first read by Splunk.
  The setting is used on a Splunk system that has configured inputs
  acquiring the data.
Highlighted

Re: Monitor log files with spaces in the file name

Contributor

Thanks.,

I have created props.conf on Universal Forwarder :

[esd:trace]
CHARSET = AUTO

but when I search the data , it is shown in binary format.

0 Karma
Highlighted

Re: Monitor log files with spaces in the file name

Ultra Champion

Why not run the UF on the windows server running the app, Maybe this would avoid mounting the share in the first place? (but maybe not)

0 Karma
Highlighted

Re: Monitor log files with spaces in the file name

Communicator

It starts binary, i've made a file with UTF-16 myself and it starts with 2 Bytes Binary...

me@myserver ✓  08:59 $ file bla-utf16.log
bla-utf16.log: Little-endian UTF-16 Unicode text, with no line terminators
[~]
me@myserver ✓  08:59 $ cat bla-utf16.log
▒▒Das ist ein Test

Have you tried to add CHARSET to your props.conf?
http://docs.splunk.com/Documentation/Splunk/7.0.1/Data/Configurecharactersetencoding

0 Karma
Highlighted

Re: Monitor log files with spaces in the file name

Contributor

Thanks.,

I have created props.conf on Universal Forwarder :

[esd:trace]
CHARSET = AUTO

but when I search the data , it is shown in binary format.

0 Karma