Solved: Re: How to Splunk the SAP Security Audit Log - Page 2

afx · ‎07-08-2019

The post question did include the answer, but then it could not be marked as an answer, therefore I pushed the content into a second post that could be marked as an answer.

afx · ‎07-15-2019

This is a write-up of my experiences trying to integrate the SAP Security Audit Log into Splunk without spending money and time getting third party adapters into SAP. I want to run this without touching SAP at all.

The information about the SAP audit log was taken from here:
https://blogs.sap.com/2014/12/11/analysis-and-recommended-settings-of-the-security-audit-log-sm19-sm...

First, you need to setup a splunk user id on the SAP servers that can read the log files, so typically it should be in group sapsys.
Of course you need to know where the log file is written to.

The SAP Security Audit log is a weird beast, it is written in UTF-16 even though it only shows simple ASCII, maybe SAP has a deal with disk manufacturers.

In the following, I assume a universal forwarder on the SAP sever which is managed by a deployment server and that the indexers are managed as cluster nodes and the basic infrastructure is already set up.

Forwarder Setup for SAP SAL:
props.conf:

[sap:sal]
CHARSET=UTF-16LE
NO_BINARY_CHECK=false
detect_trailing_nulls = false

inputs.conf:

[monitor:///sapmnt/AMP/audit/SAL/*/audit_a01amp_AMP_*_000001]
index = amp_sal
sourcetype = sap:sal

The forwarder already needs to know about the UTF-16LE encoding otherwise you might get rather strange results.
The SAL has 200 character records, no proper line ends.
Therefore, we need to trick Splunk into seeing lines.
This works by using the report type indicator at the beginning (always 2 or 3 on our systems) and the two dummy 0 bytes after date and time. It will consume the initial digit of the record but this is usually not relevant for the analysis.

So now that we can send the records to the indexer we need to help Splunk to identify the fields.
The initial thought of using just fixed fields never really worked the results where unusable for analysis.
Thanks to this post: https://answers.splunk.com/answers/78772/fixed-width-data-streamsfield-extractions.html
I got the idea to split the records into delimited fields.

This needs to be done on the indexer.
Therefore, the props.conf file that gets pushed to the indexers looks similar to the one on the forwarder with one crucial entry added:
TRANSFORMS=add_separators. And the character set information is removed:

[sap:sal]
category = Custom
BREAK_ONLY_BEFORE_DATE =
LINE_BREAKER = ([23])[A-Z][A-Z][A-Z0-9]\d{14}00
TIME_PREFIX=\w{3}
TIME_FORMAT=%Y%m%d%H%M%S
MAX_TIMESTAMP_LOOKAHEAD = 14
SHOULD_LINEMERGE = false
TRANSFORMS=add_separators

The Transforms entry of course also requires a transforms.conf on the indexer, it looks like this:

[add_separators]
DEST_KEY=_raw
SOURCE_KEY=_raw
REGEX = ^(.{3})(.{8})(.{6})(\w\w)(.{5})(.{5})(.{2})(.{8})(.{12})(.{20})(.{40})(.{3})(.)(.{64})(.{20})
FORMAT=$1|$2|$3|$4|$5|$6|$7|$8|$9|$10|$11|$12|$13|$14|$15

This splits the incoming records into fields separated by pipe symbols which can then be used to have reliable field definitions.

Now that we do have the data in a parse able format, we can put together an app for the Search Head:
props.conf:

[sap:sal]
category = Custom
REPORT-SAP-Delim = REPORT-SAP-Delim

EXTRACT-SAL-A = ^.{128}(?<FA>.*?[^&\|]*)&*
EXTRACT-SAL-B = ^.{128}.*?&(?<FB>.*?[^&\|]*)&*
EXTRACT-SAL-C = ^.{128}.*?&.*?&(?<FC>.*?[^&\|]*)&*
EXTRACT-SAL-D = ^.{128}.*?&.*?&.*?&(?<FD>.*?[^\|]*)

LOOKUP-auto_sap_sm20 = sap_sm20 message_id AS message_id OUTPUTNEW audit_class AS sap_audit_class event_class AS sap_event_class message AS sap_message new_in_release AS sap_new_in_release

The fixed fields will be defined through a transformation via REPORT-SAP-Delim.
In addition, the EXTRACT statments get dynamic subfields of the message field.
Finally, the LOOKUP uses the message_id to provide some additional explanatory fields.
The table in the SAP blog post referenced above was used to create that lookup.

The fields are defined in transforms.conf:

[REPORT-SAP-Delim]
DELIMS = "|"
FIELDS = "message_id","date","time","dummy","process_id","task","proctype","term","user","transaction","app","client","sglmode","message","src"

[sap_sm20]
batch_index_query = 0
case_sensitive_match = 1
filename = SAP_SM20.csv

Now we finally have a splunkable SAP Security Audit Log.

I tried to use the four dynamic fields to fill in the placeholders in the messages via an EVAL statement in props.conf, but that will not work with the message coming out of a lookup.
On the other hand, I will probably only need this in a report, so it can be done there.
This is an example report to mimic a regular SAP report for client 100:

index=amp_sal client=100  
| eval Message=sap_message
| eval Message=replace(Message,"&A",if(FA!="",FA," ")) 
| eval Message=replace(Message,"&B",if(FB!="",FB," ")) 
| eval Message=replace(Message,"&C",if(FC!="",FC," ")) 
| eval Message=replace(Message,"&D",if(FD!="",FD," ")) 
| lookup dnslookup clientip as src OUTPUT clienthost as src_resolved
| fillnull src_resolved value="N/A" 
| convert timeformat="%Y-%m-%d %H:%M:%S" ctime(_time) as Timestamp 
| table Timestamp user client sap_event_class src_resolved transaction app message_id Message 
| rename client as Mandant src_resolved as Source user as User transaction as Transaction app as Program sap_event_class as "SAP Event Class"

This report can then easily adapted to find failed logins (message_id IN (AU2,AU6,AUO,AUM,BUD) or other relevant events.

Happy Splunking
afx

View solution in original post

becksyboy · ‎02-07-2020

Thanks afx appreciate your input. SME's for SAP here mentioned it was a migration from sm19. I'll need to go back to them to confirm if they can change the audit log format to the what it was or clarify what it is now.

I'm not sure exactly when the events start and finish as they come in all on one line, and are petty huge, extract below.

Was thinking of adding this, as events seem to follow this pattern - AUW20200128000002 etc...
BREAK_ONLY_BEFORE = [A-Z]{3}\d{4}\d{2}\d{2}\d{6}

If i add TRUNCATE = 2000000 i get to see even more of the event.

5SAL_SAP_19720607_000000_FFFFFFFFFFFF18F43420D2543289875A6507ACD3B9DDA6DC7F5C58D7327CE17A6747698BBED60035AUW20200128000002009175300111B6000100000006SAPSYS00000008RSBTCRTE000000000009RSBTCRTE&009019A8A9C1C68EAE2D5AEFC55F369A4A89B74F648AAAA8DEE58BC0A6F9FAC4ED2D0035AUW20200128000003008061100109B6000100000006SAPSYS00000008RSBTCRTE000000000009RSBTCRTE&0090746CBE5A98425B4DEAD04A50436321120C49F2826F3B755C40B5A5AEDE61300D0035AUW20200128000004008061100109B6000100000006SAPSYS00000008RSBTCRTE000000000009RSBTCRTE&00901C2FC4B044403F80521F6FD3DD1156F60F1F72A98E449FFC4D641176D26DBC530035

afx · ‎02-07-2020

From what you have shown me, the line breaker should be the same, but the events might be longer and then it gets confusing.
Would you have a test file for me to look at?

cheers
afx

becksyboy · ‎02-07-2020

thanks, ive attached a sample

jet1276 · ‎10-04-2020

Hey @afx @becksyboy,

Do we have any solution for the same? I am also getting similar issue in my SAP environment.

afx · ‎10-04-2020

Sorry, no idea.

To me the posted example feels like it was preprocessed somehow.

And from the SAP blog post, it should not matter whether RSAU or SM19 control.

How to Splunk the SAP Security Audit Log

Announcing Scheduled Export GA for Dashboard Studio

Extending Observability Content to Splunk Cloud

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!