Windows Event Message Field - RegEx Issue - Certainly something wrong / bug

[jlvix1]

Plenty of people struggle with this and with no definitive answer either... Unless someone cares to point something out I've missed from 2 hours of googling?

Take for example, security log Event ID 4625 from windows server 2016, via LF to splunk 7.0.3.

Regex using rex command on _raw (Works😞 Message\=(?<Message2>.+)\.

Regex using web GUI field extraction on this source type is identical but NOT working.

When checking the difference between my Message and Message2 fields - most are identical, occasionally the last line of text is missing from Message2.

this isn't working and I don't think it ever has, given the simplicity of this regex extraction... I've tried adding (\.|\r\n|\r|\n|\0) and had the same results.

Strange thing is, this works for other event ID's, like 4738 (A user account was changed). or 4724 (An attempt was made to reset an account's password).

I'm theorizing that splunk subsystem is possibly responsible by not correctly rebuilding events in memory when inflating?

[splunknewbie81]

maybe you want to try this -

| rex "Message="+(?<Info>.*)

Hope it works

[niketn]

Can you try with Dot (.) to match newline character as well using (?ms)?

Message\=(?ms)(?<Message2>.+)\.

Also what does the result look like if you try the following before applying rex?

 eval Message2=replace(Message2,"\n\r","")

[jlvix1]

Hi mate, tried this and none of it works, I still believe this to be an issue.

After using RegEx101.com and the (?ms) option, that seems to cause a similar issue to what I'm seeing in splunk, it picks up the last period in the message as the terminator, strange, still researching the ?ms combination.

I've tried using props.conf and still getting the same result.

I believe this is to do with the options of regex (e.g regex../options). the rex command defaults are perfect for this but the queue regex is set differently, perhaps I need to enable /g ensuring nothing else is enabled, as per regex101.com where my results are fine

[niketn]

@jlvix1 can you put the sample event? Mock/anonymize any sensitive information.

[jlvix1]

Hi niketnilay, have you had any thoughts on this? anyone???

[akocak]

Hi Jlvix,
I have the same issue here. I even tried some tricks like, saving Message to another variable and regex in that variable instead of original event, still nothing changes. I guess in the meaning, there is no solution to this bug as of now after a year.

[jlvix1]

You see pal, if I just use normal regex (Message\=(?<Message2>.+)\.) on regex101.com I get the right extraction, but only on the website, this doesn't work in splunk, even with the extra hacks we have spoken of...

This is quite frustrating... I have copied that event from my search results in splunk and not from the windows event log.

[jlvix1]

04/17/2018 12:42:30 PM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4625
EventType=0
Type=Information
ComputerName=WIN-JPMM7KL7IDE
TaskCategory=Logon
OpCode=Info
RecordNumber=45629
Keywords=Audit Failure
Message=An account failed to log on.

Subject:
    Security ID:        S-1-5-18
    Account Name:       WIN-JPMM7KL7IDE$
    Account Domain:     WORKGROUP
    Logon ID:       0x3E7

Logon Type:         7

Account For Which Logon Failed:
    Security ID:        S-1-0-0
    Account Name:       Administrator
    Account Domain:     WIN-JPMM7KL7IDE

Failure Information:
    Failure Reason:     Unknown user name or bad password.
    Status:         0xC000006D
    Sub Status:     0xC000006A

Process Information:
    Caller Process ID:  0x3ac
    Caller Process Name:    C:\Windows\System32\svchost.exe

Network Information:
    Workstation Name:   WIN-JPMM7KL7IDE
    Source Network Address: 127.0.0.1
    Source Port:        0

Detailed Authentication Information:
    Logon Process:      User32 
    Authentication Package: Negotiate
    Transited Services: -
    Package Name (NTLM only):   -
    Key Length:     0

This event is generated when a logon request fails. It is generated on the computer where access was attempted.

The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network).

The Process Information fields indicate which account and process on the system requested the logon.

The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

The authentication information fields provide detailed information about this specific logon request.
    - Transited services indicate which intermediate services have participated in this logon request.
    - Package name indicates which sub-protocol was used among the NTLM protocols.
    - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.

[RedStapler]

I was having trouble with this event message in that I wanted to RegEx out Account name and there are two of them. I have be been trying to get only the 2nd one but I get both of them. 

Account Name: WIN-JPMM7KL7IDE$
Account Name: Administrator

| rex field=Message "Account Name: \s+(?<Account_Name>[^:]+)"

[ITWhisperer]

If they are both in the same event and you only want the last one, try this:

| rex "(?ms).*Account Name:\s+(?<Account_Name>\S+)"

[jlvix1]

Sure I will get back to you man, need to do it tonight as I'm on a metaframe desktop currently

[jlvix1]

I think I am seeing some slight improvement by using the following regex...

(?m)^Message\=(?<Message2>.+)\.$ but this doesn't work for event code 4625 in particular, and windows filtering platform events, fine for all the others though...

Without the period terminator on the end, seems to work for scheduled task events... So those are now fixed.

No idea what the problem is here, I think it's the windows events containing invisible characters or mixtures of \r and \n etc...

Still baffled, not uncommon to see these sorts of issues come up and make what is expected to be a simple task a mission to mars!

[RedStapler]

I am searching the Windows Event, 'Message' body and trying to get the 2nd 'Account Name:' value only. 

Maybe the problem is that 'Account Name:' has two different values after it. The first is always a computer object and the second is the user or service account. Most of what I have tried gets me only the first value, or both the first and second. Getting only the 2nd value has evaded me. 

It seems like it would be a common hurdle to encounter with Windows events since a high percentage of the events have both. 

Message=An account failed to log on.

Subject:
Security ID: S-1-5-18
Account Name: ComputerAccount$
Account Domain: Domain
Logon ID: 0x3E8

Logon Type: 3

Account For Which Logon Failed:
Security ID: S-1-0-0
Account Name: svc_account
Account Domain: Domain

Failure Information:
Failure Reason: Account locked out.
Status: 0xC0000236
Sub Status: 0x0

++++

Failed attempts thus far: 

| rex "(?ms)^Message\=(?<Message2>.+)\.$"

| rex "(?ms).*Account Name:\s+(?<Account_Name>\S+)"

| rex "(?ms)^Message\=(?<Message2>.+)\.$"

| rex "(?ms).*Account Name:\s+(?<Account_Name>\S+)"

| rex field=Message max_match=2 "Account Name: (?.+)"
| eval logon=if(mvcount(logon)=2, mvindex(logon, 1), logon)

| rex field=Message "Account Name\:\s+(?<ACCOUNT_NAME>.+)"

| rex field=Message "Account Name:\:\s+(?<ACCOUNT_NAME>[^\r\n]+)"

| rex field=Message "Account Name:\s+(?<ACCOUNT_NAME>[^\r\n]+)"

| rex field=Message "Account Name:\s+(?<ACCOUNT_NAME>[^:]+)"

[ITWhisperer]

Here is a runanywhere example of it working

| makeresults
| eval _raw="Message=An account failed to log on.

Subject:
Security ID: S-1-5-18
Account Name: ComputerAccount$
Account Domain: Domain
Logon ID: 0x3E8

Logon Type: 3

Account For Which Logon Failed:
Security ID: S-1-0-0
Account Name: svc_account
Account Domain: Domain

Failure Information:
Failure Reason: Account locked out.
Status: 0xC0000236
Sub Status: 0x0"
| rex "(?ms).*Account Name:\s+(?<Account_Name>\S+)"

This assumes the events are as you have shown i.e. no extra characters such as quotes or other punctuation.

[jlvix1]

Hi, I will have to try this tonight as my dev workstation is at home.

Are you sure about \n\r? I thought it was \r\n? Will try both combinations anyway.

[niketn]

Well I think I have tried \n\r in past. So do try it.

[jlvix1]

Remember rex works fine its just inconvenient to have to have this in every query - because I have lots of event codes that are affected by this, and a few alerts to make based on these codes. The difference between using the web GUI and rex is ringing alarm bells, as my main point is that the web GUI doesn't work.

[jlvix1]

I think I did try both, I've tried everything though, apart from props method as opposed to using the GUI field extraction.