Why is splunkd.log not getting indexed? Receiving error "The file 'E:\Splunk\var\log\splunk\splunkd.log' is invalid. Reason: binary"

[scottrunyon]

My splunkd.log is being flooded with the following messages over and over -

01-04-2017 01:05:31.133 -0600 WARN  FileClassifierManager - The file 'E:\Splunk\var\log\splunk\splunkd.log' is invalid. Reason: binary
01-04-2017 01:05:31.133 -0600 INFO  TailReader - Ignoring file 'E:\Splunk\var\log\splunk\splunkd.log' due to: binary
01-04-2017 01:05:31.164 -0600 WARN  FileClassifierManager - The file 'E:\Splunk\var\log\splunk\splunkd.log' is invalid. Reason: binary
01-04-2017 01:05:31.164 -0600 INFO  TailReader - Ignoring file 'E:\Splunk\var\log\splunk\splunkd.log' due to: binary
01-04-2017 01:05:31.195 -0600 WARN  FileClassifierManager - The file 'E:\Splunk\var\log\splunk\splunkd.log' is invalid. Reason: binary
01-04-2017 01:05:31.195 -0600 INFO  TailReader - Ignoring file 'E:\Splunk\var\log\splunk\splunkd.log' due to: binary

I am running Splunk Enterprise 6.5.0. This system is half of an indexer cluster and the other system in the cluster is not getting these messages.

[supabuck · Accepted solution]

Hello,

I think for some reason it believes that it is a binary file rather than ascii. I recommend stopping Splunk, copy the contents of it, delete the file and create a new file with that name with appropriate permissions in the $SPLUNK_HOME/var/log/splunk/ directory then paste back in the plain text to your new file and restart Splunk.

Regards,
supabuck

[ddrillic]

Similar thread at splunk thinks text file is binary

[supabuck]

Hello,

I would try to stop the splunk process on that host, move the splunkd.log file to another name in the same directory such as splunkd.log.txt and let splunk re-create the file as it should be. I think for some reason it believes that it is a binary file rather than ascii. You could also probably just copy the contents of it, delete the file and create a new file with that name with appropriate permissions in the $SPLUNK_HOME/var/log/splunk/ directory then paste back in the plain text to your new file and restart Splunk.

Let me know if this works.

Regards,
supabuck

[scottrunyon]

The splunkd.log has rolled and it looks like the problem is solved by creating the new file.

Thank you for the help.

[supabuck]

That's great! Would you mind accepting the answer below?

[scottrunyon]

I renamed the splunkd.log file and started Splunk. This did not clear the messages.

I rename splunkd.log again, created a new file and the messages stopped.

The log shows that both splunk.log and btool.log plus the archived files (.1, .2, etc) are all binary. I created a new btool.log file and that appears to be cleared as well.

Any idea of how they could have been changed? I am concerned that when the current files roll to .1, the new file will be returned to binary.

Runiing Splunk Enterpirse 6.5.0 on Windows 2008 server.

[supabuck]

In this case, I'm not too sure. I would open a case with Splunk to see if they have ever seen this issue. The answer below also has a valid situation but it doesn't explain how it was created which I am unsure of.