Splunk Search

Rex Field Question

johnblakley
Explorer

I have a message field in an event id that isn't extracting properly. The part I've having an issue with is when there's a special character in the middle (of unknown length string) of the string.

For example, I can extract "test\user" with no issues, but when I have something like below:

test\user.name
test\user-name
test\username-was-here
test\username-was-not.here$

I'm unable to extract them. I've tried:

rex field=Message "(?<SubjectID>[1-9A-Za-z$].+[-$._].+[1-9A-Za-z$]

I've tried the $ with the \$ as well, but this doesn't provide any results. I've also tried taking the 'test\user-name' field and plug it in directly to see if it was show me a result, but that failed to work as well. Is there a way to do this where I could take into account all scenarios above?

Here's the log entry:

alt text

Below is a message that doesn't work due to spaces in the name/ID:

alt text
Thanks!

Tags (1)
0 Karma

niketn
Legend

[Updated Answer]

Since Security ID: is followed by Account Name:, following regex extracts all characters between the two using .+

<yourBaseSearch> EventCode=4724 Message="*" "Subject:" "Target Account:" "Security ID:"
|  rex field=Message "Security ID:\s+(?<SecurityID>.+)\s+Account Name:" max_match=2
|  eval SubjectSecurityID=mvindex(SecurityID,0), TargetAccountSecurityID=mvindex(SecurityID,1)

Please try out and confirm!


@johnblakley, please try the following:

<yourBaseSearch> EventCode=4724 Message="*" "Subject:" "Target Account:" "Security ID:"
|  rex field=Message "Security ID:\s+(?<SecurityID>[^\s]+)\s" max_match=2
|  eval SubjectSecurityID=mvindex(SecurityID,0), TargetAccountSecurityID=mvindex(SecurityID,1)
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

johnblakley
Explorer

This works, but it's also breaking at the space in the security ID.

0 Karma

niketn
Legend

I have updated the Regular Expression please try out and confirm!

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

niketn
Legend

@johnblankley, which field are you trying to extract?
It is Security ID or Account Name or Logon ID? Also is it from Subject or from Target Account?

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

johnblakley
Explorer

Security ID from both Subject and Target sections.

0 Karma

niketn
Legend

Do you want to extract them as multi-value in the same field or separate fields?

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

493669
Super Champion

can you provide whole raw data (with important data to be masked) to understand starting and ending to extract id

0 Karma

johnblakley
Explorer

This is very close! I had to modify it a little, but I noticed a new issue. How can I take into account the SubjectSecurityID to have a space? What I'm seeing is "NT AUTHORITY\SYSTEM" only shows "NT". I've played around with adding something like "(?\S+\s+\w+), but that's not working.

0 Karma

493669
Super Champion

Try this:

...| rex field=Message  "(?ms)Subject:\nSecurity ID:(?<SubjectSecurityID>.*)Account Name.*Target Account:\nSecurity ID:(?<TargetSecurityID>.*)Account Name"
0 Karma

johnblakley
Explorer

Unfortunately, that didn't work. It looks like the message field is one line of characters, so the Subject now becomes the full message when using .*

The result is this:
NT AUTHORITY\SYSTEM Account Name: xxxx$ Account Domain: xxxx Logon ID: 0x3e7

It should just be:
NT AUTHORITY\SYSTEM

0 Karma

493669
Super Champion

can you provide whole message where it won't work...

0 Karma

johnblakley
Explorer

This seems to have worked...do you see any issues with it?

(?\S+.\w+.\w+)

0 Karma

493669
Super Champion

please use 101010 button for query so that no special characters get removed.
it will work only for particular pattern.
have you tried below...it seems to be working..

...| rex field=Message  "(?ms)Subject:\nSecurity ID:(?<SubjectSecurityID>.*)Account Name.*Target Account:\nSecurity ID:(?<TargetSecurityID>.*)Account Name"
0 Karma

johnblakley
Explorer

Added to original post. It's breaking on security IDs with spaces. This is just one example, so the \S+ is stopping at the space.

0 Karma

johnblakley
Explorer

Added to original post...thanks!

0 Karma

493669
Super Champion

so here are you trying to extract security id?

0 Karma

johnblakley
Explorer

Yes for both Subject and Target accounts. My regex works on test sites to capture all special characters, but Splunk doesn't work. It's simple enough to do "\w+[-].+[-].+" and it will find anything with two dashes. Splunk won't complain, but it will return a blank result with rex for the extracted field in

0 Karma

493669
Super Champion

try this for Subject accounts

...|rex field=raw "Subject:\nSecurity ID:(?<SubjectSecurityID>\S+)"
0 Karma

johnblakley
Explorer

Okay, thank you! Field=raw doesn't work for some reason, but this seems to:

... | rex field=Message "Subject:\s+Security\s+ID:\s+(?<SubjectSecurityID>\S+)" | table SubjectSecurityID
0 Karma

493669
Super Champion

now try this for both:

...| rex field=Message "(?ms)Subject:\nSecurity ID:(?<SubjectSecurityID>\S+).*Target Account:\nSecurity ID:(?<TargetSecurityID>\S+)"
0 Karma
Get Updates on the Splunk Community!

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...

Avoid Certificate Expiry Issues in Splunk Enterprise with Certificate Assist

This blog post is part 2 of 4 of a series on Splunk Assist. Click the links below to see the other ...

Using Machine Learning for Hunting Security Threats

REGISTER NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more ...