Splunk Search

rex over 2 lines with 2 combined search phrases

Stonecore
New Member

I'm facing a problem with rex and working through many many threads which didn't help me to solve this issue.

I have logs with attribute based xml (xacml) and want to extract a value, which can be found after 2 special key-phrases in 2 lines.
Here is an log example, which should help to understand this issue.
example

Well, I can get every value by this simple rex-command.
rex max_match=0 "string\">"(?<parameter>.*?)</xacml-ctx:AttributeValue>


But I only need value2, which can clearly identified by the name "function2" in the line above.

By other examples i know this rex command should start with:
rex field=_raw ".*function2(?:\n|.)*
The expression (?:\n|.)* matches any sequence of characters, including a newline.

Next is a kind of verify/bridge-statement. (?=string\">)
The expression (?=string\">) checks that the previous expression is followed by string\"> , without "eating up" the match, so it is left for the next expression to pick up

And last my known search and extract string.
"string\">"(?<parameter>.*?)</xacml-ctx:AttributeValue>

But putting all parts together doesn't work.
rex field=raw ".*function2(?:\n|.)*(?=string\">)"string\">"(?<parameter>.*?)</xacml-ctx:AttributeValue>"

What's the reason? Could it be a simple probleme of connecting statements together?
alt text

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

@woodcock's answer should work fine. If not, here's another way of looking at it. Pull out the parameters as a multivalue field, pull out the attributes as a multivalue field, zip them together, then extract only the one combo we want.

| rex max_match=0 "string\">"(?<parameter>.*?)\</xacml-ctx:AttributeValue>" 
| rex max_match=0 "AttributeId\"(?<myID>[^\"]+)\" "
| eval temp=mvzip(myID,parameter,"!!!!") 
| rex field=temp "^function2!!!!(?<parameter>.*)"  

Initially I had an mvfilter command in there to kill the ones we didn't want, but then I realized it wasn't needed because we could code the rex to extract only the one we wanted.

0 Karma

woodcock
Esteemed Legend

Try this:

... | rex "(?ms)AttributeID=\"function2\".*?string\">(?<parameter>.*?)<\/xacml-ctx:AttributeValue>"

The key is the (?ms) part which says to work multi-line and to consider newlines as matching the . wildcard.

Stonecore
New Member

hey woodcock, also thanks for your reply and input.
I got it running and correct working with a shorter version based on your input.
| rex "(?s)function2.*?string\">(?<_parameter>[^<]+)" (without _)

0 Karma

woodcock
Esteemed Legend

Yes, the m part is implied by default when using rex in splunk, so it is redundant, but it helps to have it there for commentary's sake (and so that it works when you use other tools such as regex101.com, etc.). Be sure to upvote any helpful answers and click Accept on the best one.

0 Karma

dineshraj9
Builder

Try this way -

| rex max_match=0 "function2.+string\"\>(?<parameter>[^\<]+)"
0 Karma

Stonecore
New Member

Hey dineshraj9, thanks for your reply and input.
Your statement didn't worked, but it helped me to shorten the end.
And i tried some other modifier based on the core of your statement
and finally i got one working, but often with incorrect results.
| rex (?s)"function2(p:\n|.).+string\">(?<_parameter>[^<]+)" (without _)

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...