Splunk Search

Windows Event Message Field - RegEx Issue - Certainly something wrong / bug

Communicator

Plenty of people struggle with this and with no definitive answer either... Unless someone cares to point something out I've missed from 2 hours of googling?

Take for example, security log Event ID 4625 from windows server 2016, via LF to splunk 7.0.3.

Regex using rex command on _raw (Works😞 Message\=(?<Message2>.+)\.

Regex using web GUI field extraction on this source type is identical but NOT working.

When checking the difference between my Message and Message2 fields - most are identical, occasionally the last line of text is missing from Message2.

this isn't working and I don't think it ever has, given the simplicity of this regex extraction... I've tried adding (\.|\r\n|\r|\n|\0) and had the same results.

Strange thing is, this works for other event ID's, like 4738 (A user account was changed). or 4724 (An attempt was made to reset an account's password).

I'm theorizing that splunk subsystem is possibly responsible by not correctly rebuilding events in memory when inflating?

0 Karma

SplunkTrust
SplunkTrust

Can you try with Dot (.) to match newline character as well using (?ms)?

Message\=(?ms)(?<Message2>.+)\.

Also what does the result look like if you try the following before applying rex?

 eval Message2=replace(Message2,"\n\r","")
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

Communicator

Hi mate, tried this and none of it works, I still believe this to be an issue.

After using RegEx101.com and the (?ms) option, that seems to cause a similar issue to what I'm seeing in splunk, it picks up the last period in the message as the terminator, strange, still researching the ?ms combination.

I've tried using props.conf and still getting the same result.

I believe this is to do with the options of regex (e.g regex../options). the rex command defaults are perfect for this but the queue regex is set differently, perhaps I need to enable /g ensuring nothing else is enabled, as per regex101.com where my results are fine

0 Karma

SplunkTrust
SplunkTrust

@jlvix1 can you put the sample event? Mock/anonymize any sensitive information.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

Communicator

Hi niketnilay, have you had any thoughts on this? anyone???

0 Karma

Contributor

Hi Jlvix,
I have the same issue here. I even tried some tricks like, saving Message to another variable and regex in that variable instead of original event, still nothing changes. I guess in the meaning, there is no solution to this bug as of now after a year.

0 Karma

Communicator

You see pal, if I just use normal regex (Message\=(?<Message2>.+)\.) on regex101.com I get the right extraction, but only on the website, this doesn't work in splunk, even with the extra hacks we have spoken of...

This is quite frustrating... I have copied that event from my search results in splunk and not from the windows event log.

0 Karma

Communicator
04/17/2018 12:42:30 PM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4625
EventType=0
Type=Information
ComputerName=WIN-JPMM7KL7IDE
TaskCategory=Logon
OpCode=Info
RecordNumber=45629
Keywords=Audit Failure
Message=An account failed to log on.

Subject:
    Security ID:        S-1-5-18
    Account Name:       WIN-JPMM7KL7IDE$
    Account Domain:     WORKGROUP
    Logon ID:       0x3E7

Logon Type:         7

Account For Which Logon Failed:
    Security ID:        S-1-0-0
    Account Name:       Administrator
    Account Domain:     WIN-JPMM7KL7IDE

Failure Information:
    Failure Reason:     Unknown user name or bad password.
    Status:         0xC000006D
    Sub Status:     0xC000006A

Process Information:
    Caller Process ID:  0x3ac
    Caller Process Name:    C:\Windows\System32\svchost.exe

Network Information:
    Workstation Name:   WIN-JPMM7KL7IDE
    Source Network Address: 127.0.0.1
    Source Port:        0

Detailed Authentication Information:
    Logon Process:      User32 
    Authentication Package: Negotiate
    Transited Services: -
    Package Name (NTLM only):   -
    Key Length:     0

This event is generated when a logon request fails. It is generated on the computer where access was attempted.

The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network).

The Process Information fields indicate which account and process on the system requested the logon.

The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

The authentication information fields provide detailed information about this specific logon request.
    - Transited services indicate which intermediate services have participated in this logon request.
    - Package name indicates which sub-protocol was used among the NTLM protocols.
    - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.
0 Karma

Communicator

Sure I will get back to you man, need to do it tonight as I'm on a metaframe desktop currently

0 Karma

Communicator

I think I am seeing some slight improvement by using the following regex...

(?m)^Message\=(?<Message2>.+)\.$ but this doesn't work for event code 4625 in particular, and windows filtering platform events, fine for all the others though...

Without the period terminator on the end, seems to work for scheduled task events... So those are now fixed.

No idea what the problem is here, I think it's the windows events containing invisible characters or mixtures of \r and \n etc...

Still baffled, not uncommon to see these sorts of issues come up and make what is expected to be a simple task a mission to mars!

0 Karma

Communicator

Hi, I will have to try this tonight as my dev workstation is at home.

Are you sure about \n\r? I thought it was \r\n? Will try both combinations anyway.

0 Karma

SplunkTrust
SplunkTrust

Well I think I have tried \n\r in past. So do try it.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

Communicator

Remember rex works fine its just inconvenient to have to have this in every query - because I have lots of event codes that are affected by this, and a few alerts to make based on these codes. The difference between using the web GUI and rex is ringing alarm bells, as my main point is that the web GUI doesn't work.

0 Karma

Communicator

I think I did try both, I've tried everything though, apart from props method as opposed to using the GUI field extraction.

0 Karma