How to create a Regex question for field extraction for our shibboleth:audit sourcetype events?

[jwalzerpitt]

I created the following regex to extract the fields for our shibboleth:audit sourcetype events:

^(?:[^\|\n]*\|){2}(?P<requestBinding>[^\|]+|)\|(?P<requestId>[^\|]+)\|(?P<relyingPartyId>[^\|]+)[^\|\n]*\|(?P<messageProfileId>[^\|]+)\|(?P<assertingPartyId>[^\|]+)[^\|\n]*\|(?P<responseBinding>[^\|]+)\|(?P<responseId>[^\|]+)\|(?P<principalName>[^\|]+)\|(?P<authNMethod>[^\|]+)\|(?P<releasedAttributeId1>[^\|]+)\|(?P<releasedAttributeId2>[^\|]+)\|(?P<nameIdentifier>[^\|]+)\|(?P<src>[^\|]+)\|(?P<assertion2ID>[^\|]+)

The issue is when reviewing events I have some events that have no values for some fields as they reflect user logouts, as such:

How can I modify the regex to look for empty strings so I cover both types of events? I have tried ^$| in front of my patterns for the capture groups, but that's not working.

{"EventReceivedTime":"2019-06-12 08:20:25","SourceModuleName":"SHIBAUDITPRD","SourceModuleType":"im_file","Event":"2019-06-12 08:20:25,175|20190612T122025Z||||http://shibboleth.net/ns/profiles/logout||||username|||||x.x.x.x|7740123422491E44E893B56AF9CB4990|","SyslogFacilityValue":1,"SyslogFacility":"USER","SyslogSeverityValue":5,"SyslogSeverity":"NOTICE","SeverityValue":2,"Severity":"INFO","Hostname":"hostname","EventTime":"2019-06-12 08:20:25"}

Thx

[jwalzerpitt · Accepted solution]

Well after mucking around right after I posted this question, I believe I figured it out. I added the following: |^$| before every pattern and when testing in Slunk I had no events listed under Non-Matches.

So the full regex is as follows:

^(?:[^|\n]|){2}(?P[^|]+|)|(?P|^$|[^|]+)|(?P|^$|[^|]+)[^|\n]|(?P|^$|[^|]+)|(?P|^$|[^|]+)[^|\n]*|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P|^$|[^|]+)|(?P[^|]+)

Hope this helps someone out