topic Re: Dynatrace audit logs indexing problem in All Apps and Add-ons

Dynatrace audit logs indexing problem

splunkreal — Tue, 14 Aug 2018 15:20:53 GMT

Hello,

we try to index correctly SecAudit-BackendServer.1.log from Dynatrace however the non-encrypted log files have special characters just before the timestamp :

\x00\x00\x00\xEB\x00\x00\x002018-08-14T16:34:51.920+0200 user=toto,source=1.2.3.4,category=AuditLog,object=,event=Access,status=success,message="successfully read audit log /opt/dynatrace/dynatrace-7.0/log/server/SecAudit-FrontendServer.1.log"

in ssh:

How would you handle with TIME_PREFIX in props.conf?

Thanks.

Re: Dynatrace audit logs indexing problem

sudosplunk — Tue, 14 Aug 2018 20:08:15 GMT

Hi @realsplunk,

IMO, this should be handled by using LINE_BREAKER. Configure line breaking to discard all special characters before date, something like below

props.conf

[sourcetype]
LINE_BREAKER = ([\r\n]+^.+)\d{4}\-\d{2}\-\d{1,2} ## This will discard newline, carriage return characters along with encrypted text.
SHOULD_LINEMERGE = false
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N%z

Re: Dynatrace audit logs indexing problem

splunkreal — Tue, 14 Aug 2018 22:59:22 GMT

Thanks, however as you can see there are special characters, how would you write regex with :

beginning with anything (risky?) until YYYY-MM-DD?

LINE_BREAKER = (^.+)\d{4}-\d{2}-\d{1,2} ?

https://regex101.com/r/6fN7JB/1

Thanks 🙂

Re: Dynatrace audit logs indexing problem

sudosplunk — Wed, 15 Aug 2018 12:22:43 GMT

Your LINE_BREAKING regex should also include \n-new line and \r-carriage return characters inside capturing group.

Per docs,

The LINE_BREAKER expression must contain a capturing group (a pair of parentheses that defines an identified subcomponent of the match.)

Wherever the expression matches, Splunk software considers the start of the first capturing group to be the end of the previous event, and considers the end of the first capturing group to be the start of the next event.

Splunk software discards the contents of the first capturing group. This content will not be present in any event, as Splunk software considers this text to come between lines.

That said, LINE_BREAKER = ([\r\n]+^.+)\d{4}\-\d{2}\-\d{1,2} will discard any special characters until YYYY-MM-DD.

https://regex101.com/r/6fN7JB/2

Re: Dynatrace audit logs indexing problem

splunkreal — Fri, 17 Aug 2018 14:50:34 GMT

Hi Nittala_surya, I'm in contact with support because this doesn't work maybe bad characters impact so I'm testing SEDCMD to clean data first.