topic Re: Help with Regex extraction from request in Splunk Search

Help with Regex extraction from request

siksaw33 — Tue, 10 Jan 2023 16:05:15 GMT

please help extract adsId,offerName, currentProductDescription, offerAccountToken, offerType, offerIdentifier

message={"name":"com. ","level":"info","message":"Create -->|Request identifier : 09accf30-6cf7-4e4f-a633-c19808eff766|CreateAccountOfferEnrollment.v1|REQUEST ---> {\"correlationId\":\"09accf30-6cf7-4e4f-a633-c19808eff766\",\"ccpId\":\"HA6952B\",\"callId\":\"0109\",\"adsId\":\"camar\",\"customerId\":\"63038\",\"eventType\":\"CVP-INSTANT\",\"channelIdentifier\":\"CVP\",\"lineOfBusiness\":\"CCSG\",\"offerName\":\"Additional\",\"offerIdentifier\":\"A000\",\"sourceProductCode\":\"2X\",\"currentProductIdentifier\":\"2X\",\"currentProductDescription\":\"Pl\",\"destinationProductCode\":\"2X\",\"destinationProductName\":\"Plat\",\"fulfillmentCode\":\"GNAS\",\"requestHasSupps\":true,\"offerType\":\"consumer-stand-alone-supp\",\"offerAccountToken\":\"YAS\",\"marketName\":\"US\",\"numberOfSupps\":1,\"calledInAccountToken\":\"YAS\",\"fullName\":{\"firstName\":\"M\",\"lastName\":\"C\",\"middleName\":\"A\",\"prefix\":\"\",\"suffix\":\"\"},\"communicationInformation\":{\"channel\":\"EMAIL\",\"communicationVariables\":[],\"locale\":\"en_US\",\"physicalAddress\":{\"city\":\"P\",\"state\":\"FL\",\"zipCode\":\"33\",\"lines\":[\"48 \",\"#0114\",\"\"]},\"emailAddress\":\"cru@gmail.com\",\"isoCountryCode\":\"840\"},\"enrollmentInformation\":{\"id\":\"2023\",\"is_customer_offline\":false,\"channel_received_datetime\":\"20230109T171713.842 GMT\",\"dynamic_journey\":\"DYNAMIC_INSTANT\",\"rep_id\":\"HA6952B\",\"country_code\":\"840\",\"journey\":\"INSTANT_DECISION\",\"journey_stage\":\"SUPP",\"applicants\":[{\"number\":0,\"amex_relationship\":{\"relationships\":[{\"number\":\"3726\",\"type\""CARD\"}]},\"type\""PRIMARY\"},{\"number\":1,\"type\""NONPRIMARY\",\"has_spending_limit\":false,\"is_signature_available\":true,\"has_cash_restriction\":false,\"experience_id\""829e34d6-e89f-422b-b355-811b1aa2c79c\",\"names\":[{\"language\""EN\",\"name\":{\"first\""V\",\"last\""C\"}}],\"identifiers\":[{\"system\""DELIVERY_METHOD_IDENTIFIER\",\"id\""510DELVIDP256Cn+ has_same_address_as_primary\":false,\"emboss_name\""V\",\"language\""EN\",\"birth_date\""19\",\"spending_limit\":0,\"experience_choices\":[{\"selected_id\""USA_CONSUMER \",\"feature_name\""CARD_DESIGN\"}],\"product\":{\"offer_arrangement_id\""de7c960c46c7\",\"source_code\""A0000FYC4T\",\"short_product_id\""L81\",\"sub_product_code\""2X\"},\"addresses\":[{\"type\""HOME\",\"address\":{\"line1\""7B\",\"city\""HOUSTON\",\"region\""TX\",\"postal_code\""77028-4570\",\"country\""840\"}},{\"type\""TEMPORARY_ADDRESS\",\"address\":{\"line1\""790\",\"city\""HO\",\"region\""T\",\"postal_code\""77\",\"country\""840\"}}]}]},\"misProcessId\""3016428984\"}"}

@ITWhisperer @VatsalJagani please help

Re: Help with Regex extraction from request

richgalloway — Mon, 09 Jan 2023 17:42:04 GMT

What have you tried so far?

Re: Help with Regex extraction from request

siksaw33 — Mon, 09 Jan 2023 18:13:06 GMT

Nothing. I am a noob, non-technical PM. I want to learn splunk search, if you can share a beginners link and also please help solve this that would be great. @richgalloway

Re: Help with Regex extraction from request

richgalloway — Mon, 09 Jan 2023 20:29:58 GMT

Generally, the community is more inclined to help those who have put in some effort on their own.

Fields can be extracted using the rex command. I find it easiest, especially when new, to use one rex command for each field to extract.

| rex "adsId\\\\\":\\\\\"(?<adsId>[^\\]+)"

Re: Help with Regex extraction from request

siksaw33 — Mon, 09 Jan 2023 20:48:17 GMT

@richgalloway This gives me error
Error in 'rex' command: Encountered the following error while compiling the regex 'adsId\\":\\"(?<adsId>[^\]+)': Regex: missing terminating ] for character class.
do I use the same one for other fields? just replace adsID with offerName etc?
also how did you build the regex any knowledge resource or link you can share? how do I learn that so I can be helped by the community?

Re: Help with Regex extraction from request

richgalloway — Mon, 09 Jan 2023 21:40:06 GMT

@siksaw33 wrote:
@richgalloway This gives me error
Error in 'rex' command: Encountered the following error while compiling the regex 'adsId\\":\\"(?<adsId>[^\]+)': Regex: missing terminating ] for character class.

That error means the escaping is off. The '\\":\\"' part tells me there is one too many backslashes before each quotation mark. Try

| rex "adsId\\\\":\\\\"(?<adsId>[^\\]+)"

do I use the same one for other fields? just replace adsID with offerName etc?

Yes

also how did you build the regex any knowledge resource or link you can share? how do I learn that so I can be helped by the community?

There are many resources on the Internet for learning and experimenting with regular expressions. I like regex101.com for testing expressions. Also, there are a few talks available at conf.splunk.com.

Re: Help with Regex extraction from request

siksaw33 — Mon, 09 Jan 2023 22:04:17 GMT

Thank you but I still get the same error with the new solution

@richgalloway

Re: Help with Regex extraction from request

yuanliu — Tue, 10 Jan 2023 06:18:01 GMT

As always, I strongly recommend not to use regex to handle structured data as simple strings because there is no guarantee that the string orders will not change. As much as possible, use SPL's built-in capabilities. You data looks like compliant JSON with another compliant JSON enclosed inside message.message. spath is your friend.

Assuming that for some reason, your raw data did not extract message.message, you can do the following:

Now you have all you need. Your sample data gives. (Another point: When you illustrate complex data, use code box. Lots of JSON markups got transformed into smileys which I had to convert back.)

adsId

callId

calledInAccountToken

ccpId

channelIdentifier

communicationInformation.channel

communicationInformation.emailAddress

communicationInformation.isoCountryCode

communicationInformation.locale

communicationInformation.physicalAddress.city

communicationInformation.physicalAddress.lines{}

communicationInformation.physicalAddress.state

communicationInformation.physicalAddress.zipCode

correlationId

currentProductDescription

currentProductIdentifier

customerId

destinationProductCode

destinationProductName

enrollmentInformation.applicants{}.amex_relationship.relationships{}.number

enrollmentInformation.applicants{}.amex_relationship.relationships{}.type

enrollmentInformation.applicants{}.experience_id

enrollmentInformation.applicants{}.has_cash_restriction

enrollmentInformation.applicants{}.has_spending_limit

enrollmentInformation.applicants{}.identifiers{}.id

enrollmentInformation.applicants{}.identifiers{}.system

enrollmentInformation.applicants{}.is_signature_available

enrollmentInformation.applicants{}.names{}.language

enrollmentInformation.applicants{}.names{}.name.first

enrollmentInformation.applicants{}.names{}.name.last

enrollmentInformation.applicants{}.number

enrollmentInformation.applicants{}.type

enrollmentInformation.channel_received_datetime

enrollmentInformation.country_code

enrollmentInformation.dynamic_journey

enrollmentInformation.id

enrollmentInformation.is_customer_offline

enrollmentInformation.journey

enrollmentInformation.journey_stage

enrollmentInformation.rep_id

eventType

fulfillmentCode

fullName.firstName

fullName.lastName

fullName.middleName

fullName.prefix

fullName.suffix

level

lineOfBusiness

marketName

name

numberOfSupps

offerAccountToken

offerIdentifier

requestHasSupps

offerType

requestHasSupps

sourceProductCode

camar

0109

YAS

HA6952B

CVP

cru@gmail.com

840

en_US

#0114

09accf30-6cf7-4e4f-a633-c19808eff766

63038

Plat

3726

CARD

829e34d6-e89f-422b-b355-811b1aa2c79c

false

510DELVIDP256Cn+ has_same_address_as_primary

DELIVERY_METHOD_IDENTIFIER

true

PRIMARY

NONPRIMARY

20230109T171713.842 GMT

840

DYNAMIC_INSTANT

2023

false

INSTANT_DECISION

SUPP

HA6952B

CVP-INSTANT

GNAS

info

CCSG

com.

YAS

A000

Additional

consumer-stand-alone-supp

true