How do I filter Active Directory events based on complex multi-line searches transforms.conf

[mooree]

I'm trying to filter a stream of events at a heavy forwarder before they head for our Cloud Splunk instance to reduce the data volumes. It's AD security Event log data, where the event looks like this:

10/24/2017 01:52:11 PM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4624
EventType=0
Type=Information
ComputerName=servername.emea.company.loc
TaskCategory=Logon
OpCode=Info
RecordNumber=7062543
Keywords=Audit Success
Message=An account was successfully logged on.
.
.
.
Logon Type:         3
Impersonation Level:        Delegation
New Logon:
    Security ID:        S-1-x-21-8209345-x-y-370233
    Account Name:       WSabcd123$

the 4624 eventcode covers both users and computers unfortunately, with the computer account generating 80% of the events. I only want user events.

The outcome I therefore want is if EventCode=4624 and Account name ends in $ I want the transform to route to the nullQueue, otherwise route to the indexQueue; which is the other form of the same event code where the Account name DOESN'T end in $.

The metadata keyword is in here so that i only do this for now to my test source.

here's what I tried:

[Drop_Some4624]
REGEX = (EventCode=4624)(Account Name:.{5,30}\$)
SOURCE_KEY = MetaData:Host
FORMAT=servername
DEST_KEY = queue
FORMAT = nullQueue

[Keep_4624]
REGEX = EventCode=4624
DEST_KEY = queue
FORMAT = indexQueue

This feels like it must be possible, But...
- is it the metadata that's messing up - have I misunderstood this filter key?
- or is it the REGEX?
- or is this just too complex
- how might I approach this from another angle?

[mooree · Accepted solution]

Thanks Guys! A combination of both answers and a change to the props.conf (that i didn't share).

This is the Regex string that worked:

(?ms)(EventCode=4624).*(Account Name:.{5,30}\$)

it was then also necessary to reverse the order in the prop.conf for the two transforms: they appear to be order specific. Define the blanket rule first, then exceptions after. "Keep everything except" appears to work, rather than drop this narrow list and keep this broad list

[WinEventLog:Security]
TRANSFORMS-Filter_4624 = Keep_4624,Drop_Some4624

There was then one other change I made, as I discovered there were some cases where events were logged with both "account name" fields populated the first with the $ on the end and the second with out, which were being dropped when i wanted them
eg:

10/25/2017 09:18:41 AM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4624
EventType=0
Type=Information
ComputerName=servername.emea.company.loc
TaskCategory=Logon
OpCode=Info
RecordNumber=7206502
Keywords=Audit Success
Message=An account was successfully logged on.

Subject:
    Security ID:        S-1-5-18
    Account Name:       servername$
    Account Domain:     EMEA
    Logon ID:       0x3E7

Logon Type:         3

Impersonation Level:        Impersonation

New Logon:
    Security ID:        S-1-5-x-8209345-y-z-a
    Account Name:       account
    Account Domain:     EMEA
    Logon ID:       0x809217BA
    Logon GUID:     {00000000-0000-0000-0000-000000000000}

So I finished up with:

(?ms)(EventCode=4624).*(Account Name:).*(Account Name:.{5,30}\$)

Thanks for your help and hope this solution helps others.

[richgalloway]

@mooree, please accept an answer to help future readers find the solution.

[richgalloway]

The think the first regex is a problem. Since "Account Name" does not directly follow the event code in the event, the regex won't match. Maybe (EventCode=4624).*(Account Name:.{5,30}\$) will work.

[alemarzu]

Hi there @mooree

Try like this.

 [Drop_Some4624]
 REGEX = (?ms)^\s+EventCode=4624.+Account\sName:\s+\S+\$
 DEST_KEY = queue
 FORMAT = nullQueue

Hope it helps.