Issues Indexing SAP System Log (SM21)

[SPLAUR]

Dear splunk community,

After successfully implementing the input from @afx :

"How to Splunk the SAP Security Audit Log"

I was encouraged to implement the SAP system log (SM21) on my own.

So far, I have managed to send the log to SPLUNK, but given the log's encoding system, I am unable to process it correctly in SPLUNK.

Most likely, my error lies in the transforms.conf or props.conf.

 props.conf

[sap:systemlog]
category = Custom
REPORT-SYS = REPORT-SYS

EXTRACT-fields = ^(?<Prefix>.{3})(?<Date>.{8})(?<Time>.{6})(?<Code>\w\w)(?<Field1>.{5})(?<Field2>.{2})(?<Field3>.{3})(?<Field4>.)(?<Field5>.)(?<Field6>.{8})(?<Field7>.{12})(?<Field8>.{20})(?<Field9>.{40})(?<Field10>.{3})(?<Field11>.)(?<Field12>.{64})(?<Field13>.{20})

LOOKUP-auto_sm21 = sm21 message_id AS message_id OUTPUTNEW area AS area subid AS subid ps_posid AS ps_posid 

transforms.conf

[REPORT-SYS]
DELIMS = "|"
FIELDS = "message_id","date","time","term1","os_process_id","term2","work_process_number","type_process","term3","term4","user","term5","program","client","session","variable","term6","term7","term8","term9","id_tran","id_cont","id_cone"

 

[sm21]
batch_index_query = 0
case_sensitive_match = 1
filename = sm21.csv

Has anyone experienced a similar issue to mine? 

Best Regards.

[Dare2SplunkSAP]

If you're looking to ingest ALL sap logs, metrics, and traces, you should also investigate PowerConnect. It's amazing!

[afx]

Totally forgot to post this..

At WallSec someone put up a more complete writeup: WALLSEC IT SECURITY - SIEM Your SAP Security Audit Log with SPLUNK

Might be easier to understand for some people than my ramblings.

[afx]

Hi Splaur,

me thinks your EXTRACT-fields is not needed, that action is performed in the transforms.conf file via REPORT-SAP-Delim which refers to the line seperators generated via add_separators.

Please reread the example and stick to it also in all the names until it works. That should get you going. 

[richgalloway]

The data is a simple CSV file so the props just need to specify that.

[sap:systemlog]
INDEXED_EXTRACTIONS = csv
DATETIME_CONFIG = CURRENT

No need for REPORT or EXTRACT.

[afx]

Since when is the SAL a CSV file? It is a perverted UTF16 fixed record monstrosity.

Please read my old post on splunking the SAP log that the OP referenced to understand what is going on.

[richgalloway]

You're right.  I took the sm21.txt file in the OP to be sample data rather than a lookup table.

[afx]

Reading too fast happens to the best of us 😉