props.conf for SAP SAL / Splunk thinks it is binary

[afx]

Hi,
my props.conf for reading the SAP Security Audit Log looks like this:

[sap:sal]
category = Custom
LINE_BREAKER=.()2AU
CHARSET=utf-16be
TIME_PREFIX=2AU.
TIME_FORMAT=%Y%m%d%H%M%S
SHOULD_LINEMERGE = false
NO_BINARY_CHECK=1

But I get the following from tailreader:
05-27-2019 11:34:35.118 +0200 WARN FileClassifierManager - The file '/sapmnt/SAPK/audit/SAL/DS01/audit_01_20190527_000001' is invalid. Reason: binary.
05-27-2019 11:34:35.118 +0200 INFO TailReader - Ignoring file '/sapmnt/SAPK/audit/SAL/DS01/audit_01_20190527_000001' due to: binary

Any ideas?
thx
afx

[afx]

OK, posted a refinement here: https://answers.splunk.com/answers/757714/how-to-splunk-the-sap-security-audit-log.html

[afx · Accepted solution]

Interestingly enough, I had seen other SAP specific props.conf files that mentioned UTF-16BE and setting NO_BINARY_CHECK=true.
It turns out they where wrong.
This made it work for me:

 [sap:sal]
 category = Custom
 LINE_BREAKER=.()2AU
 CHARSET=AUTO
 TIME_PREFIX=2AU.
 TIME_FORMAT=%Y%m%d%H%M%S
 SHOULD_LINEMERGE = false
 NO_BINARY_CHECK=0

And, most important, push the props.conf file also to the forwarder!

cheers
afx

[DavidHourani]

Hi @afx,

Since you have NO_BINARY_CHECK=true it could be the encoding of the file that's causing the problem.

Check this answer out :
https://answers.splunk.com/answers/373137/why-does-splunk-think-my-file-is-binary.html

So you could try to set CHARSET=AUTO or selecting the appropriate charset from the link below:
https://docs.splunk.com/Documentation/Splunk/6.3.3/data/Configurecharactersetencoding#Comprehensive_...

Cheers,
David

[afx]

As I wrote above, standard SAP Audit Log.

$ od -A x -t x1z -v audit_01_20190527_000001 | head
000000 32 00 41 00 55 00 31 00 32 00 30 00 31 00 39 00 >2.A.U.1.2.0.1.9.<
000010 30 00 35 00 32 00 37 00 30 00 30 00 30 00 30 00 >0.5.2.7.0.0.0.0.<
000020 30 00 31 00 30 00 30 00 30 00 36 00 35 00 38 00 >0.1.0.0.0.6.5.8.<
000030 39 00 30 00 30 00 30 00 30 00 39 00 44 00 39 00 >9.0.0.0.0.9.D.9.<
000040 31 00 30 00 2e 00 34 00 32 00 2e 00 32 00 34 00 >1.0...4.2...2.4.<
000050 59 00 31 00 36 00 31 00 39 00 20 00 20 00 20 00 >Y.1.6.1.9. . . .<
000060 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 > . . . . . . . .<

[koshyk]

what's ur raw data?