This is a write-up of my experiences trying to integrate the SAP Security Audit Log into Splunk without spending money and time getting third party adapters into SAP. I want to run this without touching SAP at all. The information about the SAP audit log was taken from here: https://blogs.sap.com/2014/12/11/analysis-and-recommended-settings-of-the-security-audit-log-sm19-sm20/ First, you need to setup a splunk user id on the SAP servers that can read the log files, so typically it should be in group sapsys. Of course you need to know where the log file is written to. The SAP Security Audit log is a weird beast, it is written in UTF-16 even though it only shows simple ASCII, maybe SAP has a deal with disk manufacturers. In the following, I assume a universal forwarder on the SAP sever which is managed by a deployment server and that the indexers are managed as cluster nodes and the basic infrastructure is already set up. Forwarder Setup for SAP SAL: props.conf: [sap:sal]
CHARSET=UTF-16LE
NO_BINARY_CHECK=false
detect_trailing_nulls = false inputs.conf: [monitor:///sapmnt/AMP/audit/SAL/*/audit_a01amp_AMP_*_000001]
index = amp_sal
sourcetype = sap:sal The forwarder already needs to know about the UTF-16LE encoding otherwise you might get rather strange results. The SAL has 200 character records, no proper line ends. Therefore, we need to trick Splunk into seeing lines. This works by using the report type indicator at the beginning (always 2 or 3 on our systems) and the two dummy 0 bytes after date and time. It will consume the initial digit of the record but this is usually not relevant for the analysis. So now that we can send the records to the indexer we need to help Splunk to identify the fields. The initial thought of using just fixed fields never really worked the results where unusable for analysis. Thanks to this post: https://answers.splunk.com/answers/78772/fixed-width-data-streamsfield-extractions.html I got the idea to split the records into delimited fields. This needs to be done on the indexer. Therefore, the props.conf file that gets pushed to the indexers looks similar to the one on the forwarder with one crucial entry added: TRANSFORMS=add_separators. And the character set information is removed: [sap:sal]
category = Custom
BREAK_ONLY_BEFORE_DATE =
LINE_BREAKER = ([23])[A-Z][A-Z][A-Z0-9]\d{14}00
TIME_PREFIX=\w{3}
TIME_FORMAT=%Y%m%d%H%M%S
MAX_TIMESTAMP_LOOKAHEAD = 14
SHOULD_LINEMERGE = false
TRANSFORMS=add_separators The Transforms entry of course also requires a transforms.conf on the indexer, it looks like this: [add_separators]
DEST_KEY=_raw
SOURCE_KEY=_raw
REGEX = ^(.{3})(.{8})(.{6})(\w\w)(.{5})(.{5})(.{2})(.{8})(.{12})(.{20})(.{40})(.{3})(.)(.{64})(.{20})
FORMAT=$1|$2|$3|$4|$5|$6|$7|$8|$9|$10|$11|$12|$13|$14|$15 This splits the incoming records into fields separated by pipe symbols which can then be used to have reliable field definitions. Now that we do have the data in a parse able format, we can put together an app for the Search Head: props.conf: [sap:sal]
category = Custom
REPORT-SAP-Delim = REPORT-SAP-Delim
EXTRACT-SAL-A = ^.{128}(?<FA>.*?[^&\|]*)&*
EXTRACT-SAL-B = ^.{128}.*?&(?<FB>.*?[^&\|]*)&*
EXTRACT-SAL-C = ^.{128}.*?&.*?&(?<FC>.*?[^&\|]*)&*
EXTRACT-SAL-D = ^.{128}.*?&.*?&.*?&(?<FD>.*?[^\|]*)
EXTRACT-SAL-E = ^.{128}.*?&.*?&.*?&.*?&(?<FE>.*?[^&\|]*)&* EXTRACT-SAL-F = ^.{128}.*?&.*?&.*?&.*?&.*?&(?<FF>.*?[^\|]*)
LOOKUP-auto_sap_sm20 = sap_sm20 message_id AS message_id OUTPUTNEW audit_class AS sap_audit_class event_class AS sap_event_class message AS sap_message new_in_release AS sap_new_in_release The fixed fields will be defined through a transformation via REPORT-SAP-Delim. In addition, the EXTRACT statments get dynamic subfields of the message field. Finally, the LOOKUP uses the message_id to provide some additional explanatory fields. The table in the SAP blog post referenced above was used to create that lookup. The fields are defined in transforms.conf: [REPORT-SAP-Delim]
DELIMS = "|"
FIELDS = "message_id","date","time","dummy","process_id","task","proctype","term","user","transaction","app","client","sglmode","message","src"
[sap_sm20]
batch_index_query = 0
case_sensitive_match = 1
filename = SAP_SM20.csv Now we finally have a splunkable SAP Security Audit Log. I tried to use the six dynamic fields to fill in the placeholders in the messages via an EVAL statement in props.conf, but that will not work with the message coming out of a lookup. On the other hand, I will probably only need this in a report, so it can be done there. This is an example report to mimic a regular SAP report for client 100: index=amp_sal client=100
| eval Message=sap_message
| eval Message=replace(Message,"&A",if(FA!="",FA," "))
| eval Message=replace(Message,"&B",if(FB!="",FB," "))
| eval Message=replace(Message,"&C",if(FC!="",FC," "))
| eval Message=replace(Message,"&D",if(FD!="",FD," "))
| eval Message=replace(Message,"&E",if(FE!="",FE," "))
| eval Message=replace(Message,"&F",if(FF!="",FF," "))
| lookup dnslookup clientip as src OUTPUT clienthost as src_resolved
| fillnull src_resolved value="N/A"
| convert timeformat="%Y-%m-%d %H:%M:%S" ctime(_time) as Timestamp
| table Timestamp user client sap_event_class src_resolved transaction app message_id Message
| rename client as Mandant src_resolved as Source user as User transaction as Transaction app as Program sap_event_class as "SAP Event Class" This report can then easily adapted to find failed logins (message_id IN (AU2,AU6,AUO,AUM,BUD) or other relevant events. Happy Splunking afx
... View more