Developing for Splunk Enterprise

Regular Expression for Field extraction

c71996
Path Finder

need help with regular expression to extract CASE_ID from below request[xml]

<tns2:LogErrorActivity xmlns:tns2="http://www.abnamro.com/ErrorLogSchema" xmlns:ns="http://www.abnamro.com/capabilities/commonLogging"> <tns2:FunctionalContext> <tns2:serviceName>StabaverService</tns2:serviceName> <tns2:operationName>DistributeStabaver</tns2:operationName> <tns2:moduleName>StabaverService</tns2:moduleName> <tns2:logLevel>ERROR</tns2:logLevel> <ns:traceId>CASE_ID-pvm:0a12ne8f</ns:traceId> <tns:Keys xmlns:bw="http://www.tibco.com/bw/xpath/bw-custom-functions" xmlns:tns="http://www.abnamro.com/ErrorLogSchema" xmlns:tns1="http://www.example.com/namespaces/tns/1588822168828"> <tns:Name>CASE_ID</tns:Name> <tns:Value>pvm:0a12ne8f</tns:Value> </tns:Keys> <tns:Keys xmlns:bw="http://www.tibco.com/bw/xpath/bw-custom-functions" xmlns:tns="http://www.abnamro.com/ErrorLogSchema" xmlns:tns1="http://www.example.com/namespaces/tns/1588822168828"> <tns:Name>DOSSIER_ID</tns:Name> <tns:Value>888888000442315174</tns:Value> </tns:Keys> </tns2:FunctionalContext> <tns2:ErrorContext> <tns2:errorCode>400</tns2:errorCode> <tns2:errorMessage>{http://com/abnamro/nl/DocumentFulfillment/v3/}createDocumentFault</tns2:errorMessage> <tns:Details xmlns:bw="http://www.tibco.com/bw/xpath/bw-custom-functions" xmlns:tns="http://www.abnamro.com/ErrorLogSchema"> <tns:Detail> <tns:code /> <tns:description>{http://com/abnamro/nl/DocumentFulfillment/v3/}createDocumentFault</tns:description> </tns:Detail> </tns:Details> <tns2:ProcessStack>com.abnamro.mcc.stabaver.services.SBVDistribute/DistributeStabaverSub-&gt;com.abnamro.mcc.stabaver.subprocess.DistributeStabaverSub/DistributeStabaver-&gt;com.abnamro.SBC.CreditStore.orchestration.DistributeStabaver/GeneratePDF-&gt;com.abnamro.SBC.CreditStore.DFulfil.InvokeDocumentFulfilment/createDocument</tns2:ProcessStack> <tns:StackTrace xmlns:bw="http://www.tibco.com/bw/xpath/bw-custom-functions" xmlns:tns="http://www.abnamro.com/ErrorLogSchema"> <FaultDetails xmlns="http://tns.tibco.com/bw/fault"> <ActivityName xmlns="">createDocument</ActivityName> <Data xmlns=""> <NS1:createDocumentException xmlns:NS1="http://com/abnamro/nl/DocumentFulfillment/v3/"> <code>SC441_DOCGEN_0700</code> <description>Error in Document Generation</description> <status>Doc Not Generated</status> </NS1:createDocumentException> </Data> <Msg xmlns="">{http://com/abnamro/nl/DocumentFulfillment/v3/}createDocumentFault</Msg> <FullClass xmlns="">com.tibco.bx.core.faults.BxException</FullClass> <Class xmlns="">BxException</Class> <ProcessStack xmlns="">com.abnamro.mcc.stabaver.services.SBVDistribute/DistributeStabaverSub-&gt;com.abnamro.mcc.stabaver.subprocess.DistributeStabaverSub/DistributeStabaver-&gt;com.abnamro.SBC.CreditStore.orchestration.DistributeStabaver/GeneratePDF-&gt;com.abnamro.SBC.CreditStore.DFulfil.InvokeDocumentFulfilment/createDocument</ProcessStack> <StackTrace xmlns="">com.tibco.bx.core.faults.BxException: {http://com/abnamro/nl/DocumentFulfillment/v3/}createDocumentFault at com.tibco.bx.core.BxSystemImpl.customizeFaultDetails(BxSystemImpl.java:2076) at com.tibco.bx.core.behaviors.activity.BxCatchAllBehavior.processExtensionAttributes(BxCatchAllBehavior.java:86) at com.tibco.bx.core.behaviors.activity.BxCatchAllBehavior.doEval(BxCatchAllBehavior.java:39) at com.tibco.bx.core.behaviors.activity.BxActivityBaseBehavior.eval(BxActivityBaseBehavior.java:832) at com.tibco.bx.core.behaviors.activity.BxActivityBaseBehavior.eval(BxActivityBaseBehavior.java:1) at com.tibco.pvm.infra.api.impl.util.director.IpmEventDirectorBase.handleModelEvent(IpmEventDirectorBase.java:195) at com.tibco.pvm.infra.api.impl.util.director.IpmEventDirectorBase.handleEvent(IpmEventDirectorBase.java:111) at com.tibco.pvm.infra.support.work.impl.WorkItemAbstract.execute(WorkItemAbstract.java:177) at com.tibco.pvm.im.rt.xm.util.work.ImxSTWorkRunnable.doWorkBlock(ImxSTWorkRunnable.java:170) at com.tibco.pvm.infra.services.engine.impl.STWorkRunnable.run(STWorkRunnable.java:50) at com.tibco.pvm.im.rt.xm.util.work.ImxSTWorkRunnable.run(ImxSTWorkRunnable.java:62) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) </StackTrace> </FaultDetails> </tns:StackTrace> </tns2:ErrorContext> </tns2:LogErrorActivity>

Labels (1)
0 Karma
1 Solution

c71996
Path Finder

@woodcock  any help to extract xml name value pair from data given 

CASE_ID 

 

c71996_0-1605398655780.png

 

 

View solution in original post

Tags (2)
0 Karma

FritzWittwer1
Path Finder

Do you need the field extraction just during search time in a SPL query, in this case you could use an xpaht with an expression like //tns2:FunctionalContext/tns:Keys[child::tns:Name eq "CASE_ID"]/tns:Value

If you intend to use it in a transforms, you have to stick to the regex where others gave good answers

0 Karma

brent_weaver
Builder

Hey there! With an assumption made, the regex I came up with is 

 

^.*CASE_ID-pvm\:(?<case_id>.*)<\/ns:.*$

 

My assumption is that "pvm:" is not part of the CASE_ID field. If it is just put it inside the capture group, i.e.  inside the ( ).  

There are some great tutorials online as well as tools that can help you work through figuring out a regex, like https://regex101.com. Keep in mind Splunk is PCRE compliant, so make sure the tool you are using is as well. Regex101 allows you to select it in the upper left frame of the page. 

Learning regex is key to leveraging the power of Splunk and a very valuable skill in this industry.  Splunk can either do automatic extractions at search time, or you can use regex inline in SPL to make fields with the rex  or regex command. 

Hope this helps!

 

Tags (1)
0 Karma

c71996
Path Finder

@FritzWittwer   any help possible  on regx. inputs will be great help

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

Yo could try this

....
| rex "<tns:Name>CASE_ID<\/tns:Name>\s*<tns:Value>(?<CASE_ID>[^\<]+)"

 

And god place to test those is

https://regex101.com/r/ggLF9Z/1

r. Ismo

0 Karma

c71996
Path Finder

Could you please help with regular expression to extract BANUMBER value in below xml log (   <tns8:Name>BCNUMBER</tns8:Name>
            <tns8:Value>2003840354</tns8:Value>
)

@splu @isoutamo @FritzWittwer1 @brent_weaver 

LOg :

<tns2:LogErrorActivity xmlns:tns2="http://www.abnamro.com/ErrorLogSchema" xmlns:ns="http://www.abnamro.com/capabilities/commonLogging">
    <tns2:FunctionalContext>
        <tns2:serviceName>StabaverService</tns2:serviceName>
        <tns2:operationName>BACOListener</tns2:operationName>
        <tns2:moduleName>StabaverService</tns2:moduleName>
        <tns2:logLevel>ERROR</tns2:logLevel>
        <ns:traceId>1606124294453.2535  </ns:traceId>
        <tns2:message> Processing in BacoListner </tns2:message>
        <tns2:senderId>BACO</tns2:senderId>
        <tns8:Keys xmlns:bw="http://www.tibco.com/bw/xpath/bw-custom-functions" xmlns:tns8="http://www.abnamro.com/ErrorLogSchema" xmlns:tns3="http://www.example.org/DataFormatResource-ListenerData" xmlns:tib="http://www.tibco.com/bw/xslt/custom-functions" xmlns:ns1="http://www.abnamro.com/capabilities/commonLogging">
            <tns8:Name>BCNUMBER</tns8:Name>
            <tns8:Value>2003840354</tns8:Value>
        </tns8:Keys>
        <tns8:Keys xmlns:bw="http://www.tibco.com/bw/xpath/bw-custom-functions" xmlns:tns8="http://www.abnamro.com/ErrorLogSchema" xmlns:tns3="http://www.example.org/DataFormatResource-ListenerData" xmlns:tib="http://www.tibco.com/bw/xslt/custom-functions" xmlns:ns1="http://www.abnamro.com/capabilities/commonLogging">
            <tns8:Name>REFDATE</tns8:Name>
            <tns8:Value>20191231</tns8:Value>
        </tns8:Keys>
        <tns8:Keys xmlns:bw="http://www.tibco.com/bw/xpath/bw-custom-functions" xmlns:tns8="http://www.abnamro.com/ErrorLogSchema" xmlns:tns3="http://www.example.org/DataFormatResource-ListenerData" xmlns:tib="http://www.tibco.com/bw/xslt/custom-functions" xmlns:ns1="http://www.abnamro.com/capabilities/commonLogging">
            <tns8:Name>CASE_ID</tns8:Name>
            <tns8:Value>pvm:0a1213qc</tns8:Value>
        </tns8:Keys>
        <tns8:Keys xmlns:bw="htt

0 Karma

c71996
Path Finder

Which can match any prefix like tns1 or tns or tns8 :

@FritzWittwer1 @brent_weaver @isoutamo 

0 Karma

richgalloway
SplunkTrust
SplunkTrust
This question has an accepted answer. If that answer does not help the please post a new question.
---
If this reply helps you, an upvote would be appreciated.
0 Karma

c71996
Path Finder

Thank you 

0 Karma

c71996
Path Finder

@woodcock  any help to extract xml name value pair from data given 

CASE_ID 

 

c71996_0-1605398655780.png

 

 

View solution in original post

Tags (2)
0 Karma

c71996
Path Finder

Hi Slunk team,

 

i found this regular expression. Could you please help me validate or find issues with it . i t worked for given data.

 

is it right ? will work for all occurrences( if i replace CASE_ID with DOSSIER_ID it is fetching data. 

 

could you please help me to validate this expression

 

RegEx:  \<tns:Name>CASE_ID<\/tns:Name>\s*<tns:Value>(?<CASE_ID>[^\<]+)\<\/tns:Value>

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try this

| rex "CASE_ID-(?<CASE_ID>[^\<]+)"
---
If this reply helps you, an upvote would be appreciated.
0 Karma

c71996
Path Finder

Thank you for response . it is not working 

c71996_0-1605397171996.png

the above extraction is not expected. It should extract value from below highlighted fields

c71996_1-1605397336303.png

based on CASE_IDbetween name fields. we must get corresponding value field highlighted.

Any help possible ?

@richgalloway  @splk  @splu @splunk  @ssl_splinker @splinks  @SplunkEng 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

That requirement was not specified in the OP so, since the value is the same in both places, I used the simpler regex.  Try this

rex "tns:Name>CASE_ID\<\/tns:Name> ?\<tns:Value>(?<CASE_ID>[^\<]+)"

 

---
If this reply helps you, an upvote would be appreciated.
0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!