Splunk Search

json field extraction

prasant
Path Finder

Hi Splunk Experts,

Below is a sample event, I have below spath msg.message.details, I am trying to extract certain  fields from the details datapath. How can I extract 'msg.message.details' into fields?, I am still a newbie and learning on the go in splunk world, I am guessing to use rex, but is there a way using spath? Our index has structured other json paths eg:y has other spath eg:msg.message.header.correlationId, etc, 

{ [-]
cf_app_id: test123
cf_app_name: test
event_type: LogMessage

job_index: ebcf8d13
message_type: OUT
msg: { [-]
level: INFO
logger: UpdateContact
message: { [-]
details: Data{SystemId='null', language='English', parentSourceSystemAction='null', contactId='cf4cae75-28b3', status='Active', birthDate='1991-01-15', eventAction='Create', Accounts=[CustomerAccounts{ Case='000899', accountid='4DA4F29E', contactRelationship=ContactRelationship{expiryDate='', contactType='owner', endDate=''}}],workContact=WorkContact{faxNumber='null', mobileNumber='null', emailAddress='null', phoneNumber='null'},homeContact=HomeContact{faxNumber='null', mobileNumber='null', emailAddress='', phoneNumber='null'},businessAddress=null,personalAddress=[PersonalAddress{addressId='9205', locality='PARK', internationalPostCode='null', internationalState='null', additionalInfo='null', isPrimary='Y', streetNumberStart='null', addressType='null', status='CO', streetNumberStartSuffix='null', postalCode='765', streetNumberEnd='null', streetName='null', country='null', streetNumberEndSuffix='null', streetType='null', state='null', subAddress=SubAddress{buildingName='null', numberStart='null', addressLines=[MIL PDE,], details=[Details{value='null', detailType='null'}, Details{value='null', detailType='null'}]}}],idv=Identification{doc=License{state='null', number='null'}}}
header: { [-]
correlationId: 707000J-52f6-10df-00f3-f859-1c5ed
entityId: cf75-2b3-cb38-cef-a72ad88
entityName: test
errorCode: null
errorMessage: null
eventName: testevent
processName: process1
processStatus: SUCCESS
serviceName: testservice
serviceType: Dispatch
}
}

timestamp: 2021-07-20
}
origin: rep
timestamp: 1626764261880766200
}

Any help is much appreciated.

Thanks

Labels (1)
0 Karma
1 Solution

kamlesh_vaghela
SplunkTrust
SplunkTrust

@prasant  
Your shared JSON is invalid, may be due to you copied from the event bar. You suppose to provide _raw data. 

Something like..

{"msg": {"message": {"details": "Data{SystemId='null', language='English', parentSourceSystemAction='null', contactId='cf4cae75-28b3', status='Active', birthDate='1991-01-15', eventAction='Create', Accounts=[CustomerAccounts{ Case='000899', accountid='4DA4F29E', contactRelationship=ContactRelationship{expiryDate='', contactType='owner',  endDate=''}}],workContact=WorkContact{faxNumber='null', mobileNumber='null', emailAddress='null', phoneNumber='null'},homeContact=HomeContact{faxNumber='null', mobileNumber='null', emailAddress='', phoneNumber='null'},businessAddress=null,personalAddress=[PersonalAddress{addressId='9205', locality='PARK', internationalPostCode='null', internationalState='null', additionalInfo='null', isPrimary='Y', streetNumberStart='null', addressType='null', status='CO', streetNumberStartSuffix='null', postalCode='765', streetNumberEnd='null', streetName='null', country='null', streetNumberEndSuffix='null', streetType='null', state='null',  subAddress=SubAddress{buildingName='null', numberStart='null', addressLines=[MIL PDE,], details=[Details{value='null', detailType='null'}, Details{value='null', detailType='null'}]}}],identification=Identification{driverLicense=DriverLicense{state='null', number='null'}}}header: { [-]correlationId: 707000J-52f6-10df-00f3-f859-1c5edentityId: cf75-2b3-cb38-cef-a72ad88entityName: testerrorCode: nullerrorMessage: nulleventName: testeventprocessName: process1processStatus: SUCCESSserviceName: testserviceserviceType: Dispatch}}"}}}

 

Well, Your data looks like representation of any Programming Objects. So may be we need to extract thise fields using regex. Can you please share fields names which you expecting?

Meanwhile you can try  this?

YOUR_SEARCH
| spath path=msg.message.details output=details 
| eval _raw=details | extract pairdelim="\"{,}" kvdelim=":"
| table *

 

KV

View solution in original post

prasant
Path Finder

Thanks a lot @kamlesh_vaghela  now I am able to get the important fields with kvdelim and pairdelim to parse the required key value pairs. 

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

@prasant 

Can you please share your sample event in code format? Just use </> tool from toolbar.

KV

0 Karma

prasant
Path Finder
Hi Kamlesh,
Appriciate your prompt response. Hope below format helps, this is sample format, the structure is pretty much same as our actual event. Thanks
{ [-]
	cf_app_id: test123
	cf_app_name: test
	event_type: LogMessage
	job_index: ebcf8d13
	message_type: OUT
	msg: { [-]
     level: INFO 
     logger: UpdateContact
     message: { [-]
       details: Data{SystemId='null', language='English', parentSourceSystemAction='null', contactId='cf4cae75-28b3', status='Active', birthDate='1991-01-15', eventAction='Create', Accounts=[CustomerAccounts{ Case='000899', accountid='4DA4F29E', contactRelationship=ContactRelationship{expiryDate='', contactType='owner',  endDate=''}}],workContact=WorkContact{faxNumber='null', mobileNumber='null', emailAddress='null', phoneNumber='null'},homeContact=HomeContact{faxNumber='null', mobileNumber='null', emailAddress='', phoneNumber='null'},businessAddress=null,personalAddress=[PersonalAddress{addressId='9205', locality='PARK', internationalPostCode='null', internationalState='null', additionalInfo='null', isPrimary='Y', streetNumberStart='null', addressType='null', status='CO', streetNumberStartSuffix='null', postalCode='765', streetNumberEnd='null', streetName='null', country='null', streetNumberEndSuffix='null', streetType='null', state='null',  subAddress=SubAddress{buildingName='null', numberStart='null', addressLines=[MIL PDE,], details=[Details{value='null', detailType='null'}, Details{value='null', detailType='null'}]}}],identification=Identification{driverLicense=DriverLicense{state='null', number='null'}}}
       header: { [-]
         correlationId: 707000J-52f6-10df-00f3-f859-1c5ed
         entityId: cf75-2b3-cb38-cef-a72ad88
         entityName: test
         errorCode: null
         errorMessage: null
         eventName: testevent
         processName: process1
         processStatus: SUCCESS
         serviceName: testservice
         serviceType: Dispatch
       }
     }
     
     timestamp: 2021-07-20 
   }
   origin: rep
   timestamp: 1626764261880766200
}
0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

@prasant  
Your shared JSON is invalid, may be due to you copied from the event bar. You suppose to provide _raw data. 

Something like..

{"msg": {"message": {"details": "Data{SystemId='null', language='English', parentSourceSystemAction='null', contactId='cf4cae75-28b3', status='Active', birthDate='1991-01-15', eventAction='Create', Accounts=[CustomerAccounts{ Case='000899', accountid='4DA4F29E', contactRelationship=ContactRelationship{expiryDate='', contactType='owner',  endDate=''}}],workContact=WorkContact{faxNumber='null', mobileNumber='null', emailAddress='null', phoneNumber='null'},homeContact=HomeContact{faxNumber='null', mobileNumber='null', emailAddress='', phoneNumber='null'},businessAddress=null,personalAddress=[PersonalAddress{addressId='9205', locality='PARK', internationalPostCode='null', internationalState='null', additionalInfo='null', isPrimary='Y', streetNumberStart='null', addressType='null', status='CO', streetNumberStartSuffix='null', postalCode='765', streetNumberEnd='null', streetName='null', country='null', streetNumberEndSuffix='null', streetType='null', state='null',  subAddress=SubAddress{buildingName='null', numberStart='null', addressLines=[MIL PDE,], details=[Details{value='null', detailType='null'}, Details{value='null', detailType='null'}]}}],identification=Identification{driverLicense=DriverLicense{state='null', number='null'}}}header: { [-]correlationId: 707000J-52f6-10df-00f3-f859-1c5edentityId: cf75-2b3-cb38-cef-a72ad88entityName: testerrorCode: nullerrorMessage: nulleventName: testeventprocessName: process1processStatus: SUCCESSserviceName: testserviceserviceType: Dispatch}}"}}}

 

Well, Your data looks like representation of any Programming Objects. So may be we need to extract thise fields using regex. Can you please share fields names which you expecting?

Meanwhile you can try  this?

YOUR_SEARCH
| spath path=msg.message.details output=details 
| eval _raw=details | extract pairdelim="\"{,}" kvdelim=":"
| table *

 

KV

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...