Plenty of people struggle with this and with no definitive answer either... Unless someone cares to point something out I've missed from 2 hours of googling?
Take for example, security log Event ID 4625 from windows server 2016, via LF to splunk 7.0.3.
Regex using rex command on _raw (Works😞
Regex using web GUI field extraction on this source type is identical but NOT working.
When checking the difference between my Message and Message2 fields - most are identical, occasionally the last line of text is missing from Message2.
this isn't working and I don't think it ever has, given the simplicity of this regex extraction... I've tried adding
(\.|\r\n|\r|\n|\0) and had the same results.
Strange thing is, this works for other event ID's, like 4738 (A user account was changed). or 4724 (An attempt was made to reset an account's password).
I'm theorizing that splunk subsystem is possibly responsible by not correctly rebuilding events in memory when inflating?
Can you try with Dot
(.) to match newline character as well using
Also what does the result look like if you try the following before applying rex?
Hi mate, tried this and none of it works, I still believe this to be an issue.
After using RegEx101.com and the (?ms) option, that seems to cause a similar issue to what I'm seeing in splunk, it picks up the last period in the message as the terminator, strange, still researching the ?ms combination.
I've tried using props.conf and still getting the same result.
I believe this is to do with the options of regex (e.g regex../options). the rex command defaults are perfect for this but the queue regex is set differently, perhaps I need to enable /g ensuring nothing else is enabled, as per regex101.com where my results are fine
I have the same issue here. I even tried some tricks like, saving Message to another variable and regex in that variable instead of original event, still nothing changes. I guess in the meaning, there is no solution to this bug as of now after a year.
You see pal, if I just use normal regex
(Message\=(?<Message2>.+)\.) on regex101.com I get the right extraction, but only on the website, this doesn't work in splunk, even with the extra hacks we have spoken of...
This is quite frustrating... I have copied that event from my search results in splunk and not from the windows event log.
04/17/2018 12:42:30 PM LogName=Security SourceName=Microsoft Windows security auditing. EventCode=4625 EventType=0 Type=Information ComputerName=WIN-JPMM7KL7IDE TaskCategory=Logon OpCode=Info RecordNumber=45629 Keywords=Audit Failure Message=An account failed to log on. Subject: Security ID: S-1-5-18 Account Name: WIN-JPMM7KL7IDE$ Account Domain: WORKGROUP Logon ID: 0x3E7 Logon Type: 7 Account For Which Logon Failed: Security ID: S-1-0-0 Account Name: Administrator Account Domain: WIN-JPMM7KL7IDE Failure Information: Failure Reason: Unknown user name or bad password. Status: 0xC000006D Sub Status: 0xC000006A Process Information: Caller Process ID: 0x3ac Caller Process Name: C:\Windows\System32\svchost.exe Network Information: Workstation Name: WIN-JPMM7KL7IDE Source Network Address: 127.0.0.1 Source Port: 0 Detailed Authentication Information: Logon Process: User32 Authentication Package: Negotiate Transited Services: - Package Name (NTLM only): - Key Length: 0 This event is generated when a logon request fails. It is generated on the computer where access was attempted. The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe. The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network). The Process Information fields indicate which account and process on the system requested the logon. The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases. The authentication information fields provide detailed information about this specific logon request. - Transited services indicate which intermediate services have participated in this logon request. - Package name indicates which sub-protocol was used among the NTLM protocols. - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.
I think I am seeing some slight improvement by using the following regex...
(?m)^Message\=(?<Message2>.+)\.$ but this doesn't work for event code 4625 in particular, and windows filtering platform events, fine for all the others though...
Without the period terminator on the end, seems to work for scheduled task events... So those are now fixed.
No idea what the problem is here, I think it's the windows events containing invisible characters or mixtures of
Still baffled, not uncommon to see these sorts of issues come up and make what is expected to be a simple task a mission to mars!
Remember rex works fine its just inconvenient to have to have this in every query - because I have lots of event codes that are affected by this, and a few alerts to make based on these codes. The difference between using the web GUI and rex is ringing alarm bells, as my main point is that the web GUI doesn't work.