topic Re: Need Regex help in parsing specific fields in Splunk Search

Need Regex help in parsing specific fields

arunsubram — Tue, 29 Sep 2020 11:09:05 GMT

String is -----------------
OfferRedeemedRequest [partnerID=1234, partnerName=MCenter, messagePriority=9, userID=2a28bc-119d7597, channel=rest, offerIds=1bf6-16a0fdd59fc4, redemptionDate=2016-05-14T11:56:24+0000, productBarCode=null, productBarCodeType=null, productQuantity=null, productCost=null, productDiscount=null, transactionID=null, posID=null, storeID=null, storeName=null, correlationID=null, JmsHeaderPriority=4, JMSPriority=4, destination=PARTNER_REDEMPTION_CONNECTOR_SENDER_QUEUE, bannerOmsPid=151, bodyPayload={"offerRedemptionProductDetails":[{"productBarCode":"342423409351","productBarCodeType":"gtin","productQuantity":1,"productCost":3.99,"actualDiscount":"-5.0"}],"offerIds":"1bf8b493-e6a0fdd59fc4","timeZoneOffset":0,"appId":"30cbd0ce-9ae4-44dc-a783-1f391942874d","bannerOmsPid":18651,"eventType":"OfferRedeemed","partnerId":"14p59-11e2-8b8b-0800200c9p69","userId":"2a2b88e0-e8bc-0050569d7597","priority":9,"channel":"rest","details":{"offerIdType":"offerId","location":"","OfferGs1":"8110003025902520535001101503161231","Void":"No","ProductQty":"0","activationDate":"2016-03-13T01:37:42+0000","OfferState":"Redeemed","OfferCode":"193218370","deviceId":"","correlationId":"4090914257","HttpUserAgent":"Jakarta Commons-HttpClient/3.0.1","ProductPrice":"0.0","RedemptionDate":"2016-03-14T11:56:24+0000","TransactionId":"11297895292","ActivationId":"2db41330-e8bc-11e5-b2eb-005056b94bc4","channel":"ECOMM"},"timestamp":"Mar 14, 2016 4:58:24 AM","partnerName":"MCenter"}]

From the above I just wanted to iterate through to create a table with just the BOLDED fields like

partnerID|offerIds|redemptionDate|-------

in below examples, there is multiple offerId in a request and wanted to table in the same column. Any help would be appreciated.

Tried OfferRedeemedRequest [partnerID=(?[^']+), but I get the entire request.

Re: Need Regex help in parsing specific fields

sundareshr — Sat, 24 Sep 2016 19:57:18 GMT

Try this

base search | extract mv_add=t pairdelim="," kvdelim="=:" | table <<BOLDED FIELDS>>

http://docs.splunk.com/Documentation/Splunk/6.4.3/SearchReference/Extract

Re: Need Regex help in parsing specific fields

aljohnson_splun — Sat, 24 Sep 2016 19:58:56 GMT

Have you tried using the field extractor?

it builds the regex for you

http://docs.splunk.com/Documentation/Splunk/6.4.3/Scenarios/Extractfields

parterID and the the other fields that are obvious a=b key-value pairs should be detected automatically in smart / verbose mode.

If not, I will help you anyway because regex is love, regex is life

https://regex101.com/r/cO8sB0/1

partnerID=(?<partnerID>\d+).*offerIds=(?<offerIds>[^,]+).*redemptionDate=(?<redemptionDate>[^,]+).*productBarCode":"(?<productBarCode>[^"]+).*productCost":(?<productCost>[^,]+).*partnerId":"(?<parnterID>[^"]+).*RedemptionDate":"(?<RedemptionDate>[^"]+).*TransactionId":"(?<TransactionId>[^"]+)

Re: Need Regex help in parsing specific fields

arunsubram — Sun, 25 Sep 2016 07:45:01 GMT

Thanks a lot. The field extractor worked perfectly fine.

Re: Need Regex help in parsing specific fields

arunsubram — Mon, 26 Sep 2016 22:37:05 GMT

Hi, Extract fields did work for me as mentioned base search | extract mv_add=t pairdelim="," kvdelim="=:" | table <>
. However in a single transaction records there is duplicates of the same <> like as you in the above string OfferIds gets repeated, partnerId gets repeated.
When I try to dedup offerId, partnerId after extracting and then table, it does not work. Any help would be appreciated to remove the duplicates

base search | extract mv_add=t pairdelim="," kvdelim="=:" |dedup ActivationId,partnerId,userId,offerIds,activationType,activationDate,channel | table ActivationId,partnerId,userId,offerIds,activationType,activationDate,channel