Splunk Search

How to create a Regex question for field extraction for our shibboleth:audit sourcetype events?

jwalzerpitt
Influencer

I created the following regex to extract the fields for our shibboleth:audit sourcetype events:

^(?:[^\|\n]*\|){2}(?P<requestBinding>[^\|]+|)\|(?P<requestId>[^\|]+)\|(?P<relyingPartyId>[^\|]+)[^\|\n]*\|(?P<messageProfileId>[^\|]+)\|(?P<assertingPartyId>[^\|]+)[^\|\n]*\|(?P<responseBinding>[^\|]+)\|(?P<responseId>[^\|]+)\|(?P<principalName>[^\|]+)\|(?P<authNMethod>[^\|]+)\|(?P<releasedAttributeId1>[^\|]+)\|(?P<releasedAttributeId2>[^\|]+)\|(?P<nameIdentifier>[^\|]+)\|(?P<src>[^\|]+)\|(?P<assertion2ID>[^\|]+)

The issue is when reviewing events I have some events that have no values for some fields as they reflect user logouts, as such:

How can I modify the regex to look for empty strings so I cover both types of events? I have tried ^$| in front of my patterns for the capture groups, but that's not working.

{"EventReceivedTime":"2019-06-12 08:20:25","SourceModuleName":"SHIBAUDITPRD","SourceModuleType":"im_file","Event":"2019-06-12 08:20:25,175|20190612T122025Z||||http://shibboleth.net/ns/profiles/logout||||username|||||x.x.x.x|7740123422491E44E893B56AF9CB4990|","SyslogFacilityValue":1,"SyslogFacility":"USER","SyslogSeverityValue":5,"SyslogSeverity":"NOTICE","SeverityValue":2,"Severity":"INFO","Hostname":"hostname","EventTime":"2019-06-12 08:20:25"}

Thx

0 Karma
1 Solution

jwalzerpitt
Influencer

Well after mucking around right after I posted this question, I believe I figured it out. I added the following: |^$| before every pattern and when testing in Slunk I had no events listed under Non-Matches.

So the full regex is as follows:

^(?:[^|\n]|){2}(?P[^|]+|)|(?P|^$|[^|]+)|(?P|^$|[^|]+)[^|\n]|(?P|^$|[^|]+)|(?P|^$|[^|]+)[^|\n]*|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P[^|]+)

Hope this helps someone out

View solution in original post

0 Karma

jwalzerpitt
Influencer

Well after mucking around right after I posted this question, I believe I figured it out. I added the following: |^$| before every pattern and when testing in Slunk I had no events listed under Non-Matches.

So the full regex is as follows:

^(?:[^|\n]|){2}(?P[^|]+|)|(?P|^$|[^|]+)|(?P|^$|[^|]+)[^|\n]|(?P|^$|[^|]+)|(?P|^$|[^|]+)[^|\n]*|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P[^|]+)

Hope this helps someone out

0 Karma
Get Updates on the Splunk Community!

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Splunk Education Goes to Washington | Splunk GovSummit 2024

If you’re in the Washington, D.C. area, this is your opportunity to take your career and Splunk skills to the ...