topic Re: Field Extraction from Multi line _raw data in Splunk Search

Field Extraction from Multi line _raw data

Supriya — Fri, 13 Nov 2020 16:09:31 GMT

Hi,

I want to extract the fields Name, Version, VendorName, usesLicensing, LicenseType, ExpiractDateString, LicenseKey, SEN based on delimiter(:) from the below raw data

Could someone please help me with the query for field extraction.

Re: Field Extraction from Multi line _raw data

richgalloway — Fri, 13 Nov 2020 18:06:42 GMT

What have you tried so far?

It would help if you posted the sample event as text rather than a screenshot so we can test without having to re-type that hideous mess (no way I'm typing all of that).

Is the number of fields constant? How about the number of ":"s? We need something to key off of to find the values.

Re: Field Extraction from Multi line _raw data

Supriya — Mon, 16 Nov 2020 05:13:56 GMT

Thank you for your response!

The Query which I"m trying is:

The field names to extract are name, version, VendorName, usesLicensing, LicenseType, ExpiryDateString, LicenseKey, SEN

But I'm to extract only one row from the below Event

Event Data:(having multiple rows in single event)

"name":"version":"VendorName":"usesLicensing":"LicenseType":"ExpiryDateString":"LicenseKey":"SEN"
"Atlassian Troubleshooting and Support Tools":"1.24.1":"Atlassian":"False"::::
"Confluence Cloud Migration Assistant":"2.5.1":"Atlassian":"False"::::
"Copy Space Plugin":"2.3":"Atlassian":"False"::::
"SAML Single Sign On for Confluence":"3.5.3":"re:solution":"True":"COMMERCIAL"::"AAABbQ0ODAoPeNqVkV9rwjAUxd/zKQJ70QdL63T+gcKkFiZTO1a3p73cdbc1mKblJpG5T7+0VuYeL
YSQe5LfPef2bmeRLzHjwYT707k/mvszHqU7PvSDGdva8hMpyd80kg4D331siTojURtRqTBdbNY8F
aqQ6LZC8UTxXpomfR5VKpcWVYY8r+j62EuRjkh93uJ4i/6YuxtliZQJkHwtMlQaWUQITZslGAwbP
4NgOAimzMEMZGYLJYZrewD9w/kzHCp5UeINCBnKVvIOjfK4R6HwgMrLqpLFR5C2RYc5SNfJFT1CX
UnbFD0wEnQtbSGU9jSUUuvK+4vguRbiiKEhi0yezb67FA1veCsrNUAGqTPijCuDCpwSf9eCTl32o
d9mn7CEClBCn80/daH4yj0i1RZBsjTehm4NgvH9eDIePbBuoFdEq6QohcGvi7Y71dgONEo2m/g1W
i3WtybBxkRNQneTebGU7UHj/983Yb/LuNtnMCwCFHR9orCqkZtBlrvqerjNLSnZ71C4AhQQjh4x1
1O2SMKZihnj4mbQ2jX+bw==X02ht":"SEN-15357546"
"SAMLWrapper-Plugin":"3.5.3":"re:solution":"False"::::
"Team Calendars":"6.0.49":"Atlassian Pty Ltd":"True":"COMMERCIAL"::"AAABTg0ODAoPeNpdkF9LwzAUxd/zKQK+6MNGWwZdBwGlKzjcH3HVJ0Gu2a0LTdNykwz105t2FXSBv
Nyc/O4556r0yJcoeZzyaL6IZoso4/m+5EkUZ2yJVpLqnGqNKBEanoNGcwCyvGqJ562ptEcjkV/vk
U5INzyOwuHPFsm+LoKiaZCkAs3XSqKxyLa+eUfaVYNEDHLmAvtN/rKnaBxSR8qicOSR5YTQe1iCQ
9Ebm8TJJJ6zsN+BdFtoUKx9Dfab8weoW30JDCJ1GmEXT3sHFLaJCnQwNxKLDSgt9ICc1j3x9ojKY
I1mKtuGFSfQfrA0/tPncC8hUj9MWACEEAZCN8Vnp+hr9J5Eg/f00sbYTvnV4RAn3202xVO+uluzH
X2AUfa87n60wVd9R2YYgmb7YivCnczTLEtmMzbi/qz2RqtGOTywR0/yCBb/t5myH9wCtHUwLgIVA
I9H5o1K8FkD5GzwxpwvJeR1NEfQAhUAkR4yfmoZx9KvHtEXWsZQ8+8qaAM=X02gk":"SEN-8799244"
"techradar":"1.1":"it-economics GmbH":"False"::::

Re: Field Extraction from Multi line _raw data

Supriya — Mon, 16 Nov 2020 01:11:14 GMT

In a single event, I have multiple rows,

(having 7 colons (:) )for each row

Re: Field Extraction from Multi line _raw data

richgalloway — Mon, 16 Nov 2020 15:21:26 GMT

Consider onboarding the data as CSV (Colon-Separated Value) so Splunk handles the field extraction for you.

If you want to do it yourself then this regex should do it.

"(?<name>[^"]+)":"(?<version>[^"]+)":"(?<VendorName>[^"]+)":"(?<usesLicensing>[^"]+)":(?:"(?<LicenseType>[^"]*)")?:(?:"(?<ExpiryDateString>[^"]*)")?:(?:"(?<LicenseKey>[^"]*)")?:(?:"(?<SEN>[^"]*)")?

Re: Field Extraction from Multi line _raw data

Supriya — Fri, 20 Nov 2020 12:00:54 GMT

This regex is not working,

Could you please help me with the Line_Breaker

Re: Field Extraction from Multi line _raw data

richgalloway — Fri, 20 Nov 2020 14:10:20 GMT

When that regex is used in the rex command the embedded quotation marks must be escaped, like this:

"(?<name>[^\\\"]+)\\\":\\\"(?<version>[^\\\"]+)\\\":\\\"(?<VendorName>[^\\\"]+)\\\":\\\"(?<usesLicensing>[^\\\"]+)\\\":(?:\\\"(?<LicenseType>[^\\\"]*)\\\")?:(?:\\\"(?<ExpiryDateString>[^\\\"]*)\\\")?:(?:\\\"(?<LicenseKey>[^\\\"]*)\\\")?:(?:\\\"(?<SEN>[^\\\"]*)\\\")?

What LINE_BREAKER? This is the first mention of that. Please post a new question about it.