Regular Expression for Field extraction

[c71996]

need help with regular expression to extract CASE_ID from below request[xml]

<tns2:LogErrorActivity xmlns:tns2="http://www.abnamro.com/ErrorLogSchema" xmlns:ns="http://www.abnamro.com/capabilities/commonLogging"> <tns2:FunctionalContext> <tns2:serviceName>StabaverService</tns2:serviceName> <tns2:operationName>DistributeStabaver</tns2:operationName> <tns2:moduleName>StabaverService</tns2:moduleName> <tns2:logLevel>ERROR</tns2:logLevel> <ns:traceId>CASE_ID-pvm:0a12ne8f</ns:traceId> <tns:Keys xmlns:bw="http://www.tibco.com/bw/xpath/bw-custom-functions" xmlns:tns="http://www.abnamro.com/ErrorLogSchema" xmlns:tns1="http://www.example.com/namespaces/tns/1588822168828"> <tns:Name>CASE_ID</tns:Name> <tns:Value>pvm:0a12ne8f</tns:Value> </tns:Keys> <tns:Keys xmlns:bw="http://www.tibco.com/bw/xpath/bw-custom-functions" xmlns:tns="http://www.abnamro.com/ErrorLogSchema" xmlns:tns1="http://www.example.com/namespaces/tns/1588822168828"> <tns:Name>DOSSIER_ID</tns:Name> <tns:Value>888888000442315174</tns:Value> </tns:Keys> </tns2:FunctionalContext> <tns2:ErrorContext> <tns2:errorCode>400</tns2:errorCode> <tns2:errorMessage>{http://com/abnamro/nl/DocumentFulfillment/v3/}createDocumentFault</tns2:errorMessage> <tns:Details xmlns:bw="http://www.tibco.com/bw/xpath/bw-custom-functions" xmlns:tns="http://www.abnamro.com/ErrorLogSchema"> <tns:Detail> <tns:code /> <tns:description>{http://com/abnamro/nl/DocumentFulfillment/v3/}createDocumentFault</tns:description> </tns:Detail> </tns:Details> <tns2:ProcessStack>com.abnamro.mcc.stabaver.services.SBVDistribute/DistributeStabaverSub-&gt;com.abnamro.mcc.stabaver.subprocess.DistributeStabaverSub/DistributeStabaver-&gt;com.abnamro.SBC.CreditStore.orchestration.DistributeStabaver/GeneratePDF-&gt;com.abnamro.SBC.CreditStore.DFulfil.InvokeDocumentFulfilment/createDocument</tns2:ProcessStack> <tns:StackTrace xmlns:bw="http://www.tibco.com/bw/xpath/bw-custom-functions" xmlns:tns="http://www.abnamro.com/ErrorLogSchema"> <FaultDetails xmlns="http://tns.tibco.com/bw/fault"> <ActivityName xmlns="">createDocument</ActivityName> <Data xmlns=""> <NS1:createDocumentException xmlns:NS1="http://com/abnamro/nl/DocumentFulfillment/v3/"> <code>SC441_DOCGEN_0700</code> <description>Error in Document Generation</description> <status>Doc Not Generated</status> </NS1:createDocumentException> </Data> <Msg xmlns="">{http://com/abnamro/nl/DocumentFulfillment/v3/}createDocumentFault</Msg> <FullClass xmlns="">com.tibco.bx.core.faults.BxException</FullClass> <Class xmlns="">BxException</Class> <ProcessStack xmlns="">com.abnamro.mcc.stabaver.services.SBVDistribute/DistributeStabaverSub-&gt;com.abnamro.mcc.stabaver.subprocess.DistributeStabaverSub/DistributeStabaver-&gt;com.abnamro.SBC.CreditStore.orchestration.DistributeStabaver/GeneratePDF-&gt;com.abnamro.SBC.CreditStore.DFulfil.InvokeDocumentFulfilment/createDocument</ProcessStack> <StackTrace xmlns="">com.tibco.bx.core.faults.BxException: {http://com/abnamro/nl/DocumentFulfillment/v3/}createDocumentFault at com.tibco.bx.core.BxSystemImpl.customizeFaultDetails(BxSystemImpl.java:2076) at com.tibco.bx.core.behaviors.activity.BxCatchAllBehavior.processExtensionAttributes(BxCatchAllBehavior.java:86) at com.tibco.bx.core.behaviors.activity.BxCatchAllBehavior.doEval(BxCatchAllBehavior.java:39) at com.tibco.bx.core.behaviors.activity.BxActivityBaseBehavior.eval(BxActivityBaseBehavior.java:832) at com.tibco.bx.core.behaviors.activity.BxActivityBaseBehavior.eval(BxActivityBaseBehavior.java:1) at com.tibco.pvm.infra.api.impl.util.director.IpmEventDirectorBase.handleModelEvent(IpmEventDirectorBase.java:195) at com.tibco.pvm.infra.api.impl.util.director.IpmEventDirectorBase.handleEvent(IpmEventDirectorBase.java:111) at com.tibco.pvm.infra.support.work.impl.WorkItemAbstract.execute(WorkItemAbstract.java:177) at com.tibco.pvm.im.rt.xm.util.work.ImxSTWorkRunnable.doWorkBlock(ImxSTWorkRunnable.java:170) at com.tibco.pvm.infra.services.engine.impl.STWorkRunnable.run(STWorkRunnable.java:50) at com.tibco.pvm.im.rt.xm.util.work.ImxSTWorkRunnable.run(ImxSTWorkRunnable.java:62) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) </StackTrace> </FaultDetails> </tns:StackTrace> </tns2:ErrorContext> </tns2:LogErrorActivity>

[FritzWittwer1]

Do you need the field extraction just during search time in a SPL query, in this case you could use an xpaht with an expression like //tns2:FunctionalContext/tns:Keys[child::tns:Name eq "CASE_ID"]/tns:Value

If you intend to use it in a transforms, you have to stick to the regex where others gave good answers

[brent_weaver]

Hey there! With an assumption made, the regex I came up with is 

^.*CASE_ID-pvm\:(?<case_id>.*)<\/ns:.*$

My assumption is that "pvm:" is not part of the CASE_ID field. If it is just put it inside the capture group, i.e.  inside the ( ).  

There are some great tutorials online as well as tools that can help you work through figuring out a regex, like https://regex101.com. Keep in mind Splunk is PCRE compliant, so make sure the tool you are using is as well. Regex101 allows you to select it in the upper left frame of the page. 

Learning regex is key to leveraging the power of Splunk and a very valuable skill in this industry.  Splunk can either do automatic extractions at search time, or you can use regex inline in SPL to make fields with the rex  or regex command. 

Hope this helps!

[c71996]

@FritzWittwer   any help possible  on regx. inputs will be great help

[isoutamo]

Hi

Yo could try this

....
| rex "<tns:Name>CASE_ID<\/tns:Name>\s*<tns:Value>(?<CASE_ID>[^\<]+)"

And god place to test those is

https://regex101.com/r/ggLF9Z/1

r. Ismo

[c71996]

Could you please help with regular expression to extract BANUMBER value in below xml log (   <tns8:Name>BCNUMBER</tns8:Name>
            <tns8:Value>2003840354</tns8:Value>)

@splu @isoutamo @FritzWittwer1 @brent_weaver 

LOg :

<tns2:LogErrorActivity xmlns:tns2="http://www.abnamro.com/ErrorLogSchema" xmlns:ns="http://www.abnamro.com/capabilities/commonLogging">
    <tns2:FunctionalContext>
        <tns2:serviceName>StabaverService</tns2:serviceName>
        <tns2:operationName>BACOListener</tns2:operationName>
        <tns2:moduleName>StabaverService</tns2:moduleName>
        <tns2:logLevel>ERROR</tns2:logLevel>
        <ns:traceId>1606124294453.2535  </ns:traceId>
        <tns2:message> Processing in BacoListner </tns2:message>
        <tns2:senderId>BACO</tns2:senderId>
        <tns8:Keys xmlns:bw="http://www.tibco.com/bw/xpath/bw-custom-functions" xmlns:tns8="http://www.abnamro.com/ErrorLogSchema" xmlns:tns3="http://www.example.org/DataFormatResource-ListenerData" xmlns:tib="http://www.tibco.com/bw/xslt/custom-functions" xmlns:ns1="http://www.abnamro.com/capabilities/commonLogging">
            <tns8:Name>BCNUMBER</tns8:Name>
            <tns8:Value>2003840354</tns8:Value>
        </tns8:Keys>
        <tns8:Keys xmlns:bw="http://www.tibco.com/bw/xpath/bw-custom-functions" xmlns:tns8="http://www.abnamro.com/ErrorLogSchema" xmlns:tns3="http://www.example.org/DataFormatResource-ListenerData" xmlns:tib="http://www.tibco.com/bw/xslt/custom-functions" xmlns:ns1="http://www.abnamro.com/capabilities/commonLogging">
            <tns8:Name>REFDATE</tns8:Name>
            <tns8:Value>20191231</tns8:Value>
        </tns8:Keys>
        <tns8:Keys xmlns:bw="http://www.tibco.com/bw/xpath/bw-custom-functions" xmlns:tns8="http://www.abnamro.com/ErrorLogSchema" xmlns:tns3="http://www.example.org/DataFormatResource-ListenerData" xmlns:tib="http://www.tibco.com/bw/xslt/custom-functions" xmlns:ns1="http://www.abnamro.com/capabilities/commonLogging">
            <tns8:Name>CASE_ID</tns8:Name>
            <tns8:Value>pvm:0a1213qc</tns8:Value>
        </tns8:Keys>
        <tns8:Keys xmlns:bw="htt

[c71996]

Which can match any prefix like tns1 or tns or tns8 :

@FritzWittwer1 @brent_weaver @isoutamo 

[richgalloway]

This question has an accepted answer. If that answer does not help the please post a new question.

[c71996]

Thank you 

[c71996 · Accepted solution]

@woodcock  any help to extract xml name value pair from data given 

CASE_ID 

[c71996]

Hi Slunk team,

i found this regular expression. Could you please help me validate or find issues with it . i t worked for given data.

is it right ? will work for all occurrences( if i replace CASE_ID with DOSSIER_ID it is fetching data. 

could you please help me to validate this expression

RegEx:  \<tns:Name>CASE_ID<\/tns:Name>\s*<tns:Value>(?<CASE_ID>[^\<]+)\<\/tns:Value>

[richgalloway]

Try this

| rex "CASE_ID-(?<CASE_ID>[^\<]+)"

[c71996]

Thank you for response . it is not working 

the above extraction is not expected. It should extract value from below highlighted fields

based on CASE_IDbetween name fields. we must get corresponding value field highlighted.

Any help possible ?

@richgalloway  @splk  @splu @splunk  @ssl_splinker @splinks  @SplunkEng 

[richgalloway]

That requirement was not specified in the OP so, since the value is the same in both places, I used the simpler regex.  Try this

rex "tns:Name>CASE_ID\<\/tns:Name> ?\<tns:Value>(?<CASE_ID>[^\<]+)"