BREAK_ONLY_BEFORE_DATE variable

[szaboszilard]

Hi everyone,

I have several oracle audit logs received via syslog-ng + splunk file inputs:

Jul  8 14:44:04 192.168.160.20 Audit[11206800]: LENGTH: "323" SESSIONID:[9] "359691618" ENTRYID:[1] "1" STATEMENT:[1] "1" USERID:[7] "SYSTEMA" USERHOST:[22] "host.domain.xy" ACTION:[3] "208" RETURNCODE:[1] "0" COMMENT$TEXT:[98] "Authenticated by: DATABASE; Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.160.21)(PORT=14039))" OS$USERID:[6] "apache" DBID:[10] "3473254904" 
Jul  8 14:47:57 192.168.160.21 Audit[27328736]: LENGTH: "156" SESSIONID: "270416291" ENTRYID: "1" ACTION: "101" RETURNCODE: "0" LOGOFF$PREAD: "0" LOGOFF$LREAD: "21" LOGOFF$LWRITE: "0" LOGOFF$DEAD: "0" SESSIONCPU: "0"

My regex

 (?i)(?<_KEY_1>\S+):(\s|\[\d+\])+"(?<_VAL_1>[^"]+)"

My props.conf

[oracle_aud]
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE_DATE = false
BREAK_ONLY_BEFORE = \d\d\d\s\d\d\s\d\d:\d\d
TIME_PREFIX = ^
TIME_FORMAT = %b %d %H:%M:%S
EXTRACT-oracle_action_returncode = (?i)(?<_KEY_1>\S+):(\s|\[\d+\])+"(?<_VAL_1>[^"]+)"
LOOKUP-oracle_action = oracle_action_csv ACTION AS ACTION OUTPUTNEW ACTION_DESCRIPTION AS ACTION_DESCRIPTION
LOOKUP-oracle_returncode = oracle_returncode_csv RETURNCODE AS RETURNCODE OUTPUTNEW RETURNCODE_DESCRIPTION AS RETURNCODE_DESCRIPTION

I cant solve 2 problem.
1. Several events displayed in one line.
2. It is possible to extract source ip (192.168.160.20) and Audit fields value (11206800|27328736) with one regex expression?

I have tried this one too, but not really worked:
BREAK_ONLY_BEFORE = ^(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Okt|Nov|Dec)\s

Regards Szilard

[richgalloway]

This regex string will extract the source IP and Audit fields

(?P<source_ip>\d+\.\d+\.\d+\.\d+) Audit\[(?P<audit>[^\]]+)]

[woodcock · Accepted solution]

#1: Change to this instead:

BREAK_ONLY_BEFORE = \w{3}\s+\d+\s+\d+:\d+

#2:

EXTRACT-srcip_and_autid = (?<srcip>\d+\.\d+\.\d+\.\d+).*?Audit\[(?<Audit>\d+)

[szaboszilard]

Extracting other fields works well, Thank you all.
i think timestamps were extracted well. I can see _time field and other time/date fields:

date_hour,date_mday,date_minute,date_month,date_second,date_wday,date_year,date_zone=local

No idea why does not work...
(Replacing sourcetype to syslog in inputs.conf line merge works well.)

[woodcock]

Do you really need to linemerge? If you don't, then changing to this should prevent mis-merged lines:

 SHOULD_LINEMERGE = false

[szaboszilard]

Option "SHOULD_LINEMERGE = false" solved it.