Getting Data In

EVAL with REX with SPATH (props / transforms)

willadams
Contributor

I have a JSON file with an embedded JSON field that I am trying to extract.  I have been doing some searching and have finally come up with an SPL search that will extract the information into my relevant key pairs.  The SPL is basically

 

index=foo sourcetype=foosource
| eval log_message=replace(log_message, "\\\"\"","")
| rex "log_message\\\\\":\\\\\"(?<log_message>[^\\\]+)"
| spath input=log_message

 

I have also read that I shouldn't need the spath however if I remove this from my SPL then it doesn't extract as required.

I would like to put this into transforms but unsure how to apply the "spath".  Thoughts around my props/transforms so far is:

props.conf

REPORT-logmessage = log_message

 

transforms.conf

[log_message]
INGEST_EVAL = eval_log_message=replace(log_message=replace(log_message, "\\\"\"","")
REX = "log_message\\\\\":\\\\\"(?<log_message>[^\\\]+)"

 

Labels (2)
0 Karma

willadams
Contributor

{"log_message";"{\"version\":\"1.00000\",\"account_id\":\"12345678\",\"region\":\"myarea\",\" and so on until this blocks ends with }}".  The remainder of the log after this is parsed appropriately "log_group":"/logs","log_location":"mylogs","account_information":"38430495"} for example.

0 Karma

thambisetty
SplunkTrust
SplunkTrust

@willadams ,

I can't give you exact answer for your question, but I can direct you the approach to be taken.

let me explain what I understand from your question:

you have new json which is log_message inside json event. you want to extract fields of log_message automatically.

I have taken sample event as below : cars is json inside json event

 

{
  "name":"John",
  "age":30,
  "cars":
    { "carname":"Ford", "models":"Fiesta" }
 }

 

first, I have extracted to cars value "{ "carname":"Ford", "models":"Fiesta" }" to cars  using props.conf below. [jsontest] is my sourcetype of event.

created a report extractcars to be used in transforms.conf

props.conf

[jsontest]
EXTRACT-cars = \"cars\":\s+\{\s(?<cars>[^\}]+)
REPORT-extractcars = extractcars

and then I have extracted fields carname and models using transforms.conf

SOURCE_KEY is key to define for which field you would to apply REGEX

transforms.conf

[extractcars]
REGEX = \"(?<_KEY_1>[^\"]+)\"\:\"(?<_VAL_1>[^\"]+)
SOURCE_KEY = cars

 

————————————
If this helps, give a like below.
0 Karma

willadams
Contributor

I will give a shot but I have to do the replace first before the rex. 

0 Karma

thambisetty
SplunkTrust
SplunkTrust

can you share sample event?

————————————
If this helps, give a like below.
0 Karma
Get Updates on the Splunk Community!

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Answers Content Calendar, July Edition I

Hello Community! Welcome to another month of Community Content Calendar series! For the month of July, we will ...