rex -> spath -> field extract not working?

[bitbucket]

Hello -

My data looks like (also attached as PNG for better readability):

2021-04-28 - 22:01:14.728 - INFO : Action completed in 7.90478181839 seconds, result is { "images-deleted": 8, "images": 444, "account": "012345678901", "task": "DELETE-AMI-TASK", "metrics": { "Action": "Ec2DeleteImageAction", "Data": { "DeletedImages": 8 }, "Version": "1.0", "Type": "action", "ActionId": "aac9da60-d325-4ed5-ae30-2e11fe7a7e39" }, "deleted": { "us-east-1": [ "ami-0dfd9eee9557ffcb3", "ami-0fec918b8f4b5bf04", "ami-00b68913ba31e0590", "ami-0859ee921a1ff93d0", "ami-06bdf5c91701957a2", "ami-00945fa203dba66df", "ami-0b35e3e1f90ff9233", "ami-032006127456fba8a" ] }, "region": "us-east-1" } - ReconNum:1619647200000

I want to extract everything between the first { and the last } with rex, cast it as JSON via spath, and then pull out the value of DeletedImages.

My search string is | rex "(?<jsonData>{[^}].+})" | spath input=jsonData output=myfield path=metrics.Data.DeletedImages

But it doesn't seem to want to pull out DeletedImages.  What am I doing wrong?

[sravankaripe]

please try with 

extract command in to extract all fields splunk https://docs.splunk.com/Documentation/Splunk/8.1.3/SearchReference/Extract

ex :  for your case

... | extract pairdelim=","   , kvdelim="/":"

OR 

... | extract pairdelim=","   ,    kvdelim="\":"

[s2_splunk]

I don't see anything wrong here. Can you append 

| fields JsonData, myfield

How do you determine that the value is not extracted?

It works fine in this run-anywhere example:

| makeresults 
| eval d = "2021-04-28 - 22:01:14.728 - INFO : Action completed in 7.90478181839 seconds, result is { \"images-deleted\": 8, \"images\": 444, \"account\": \"012345678901\", \"task\": \"DELETE-AMI-TASK\", \"metrics\": { \"Action\": \"Ec2DeleteImageAction\", \"Data\": { \"DeletedImages\": 8 }, \"Version\": \"1.0\", \"Type\": \"action\", \"ActionId\": \"aac9da60-d325-4ed5-ae30-2e11fe7a7e39\" }, \"deleted\": { \"us-east-1\": [ \"ami-0dfd9eee9557ffcb3\", \"ami-0fec918b8f4b5bf04\", \"ami-00b68913ba31e0590\", \"ami-0859ee921a1ff93d0\", \"ami-06bdf5c91701957a2\", \"ami-00945fa203dba66df\", \"ami-0b35e3e1f90ff9233\", \"ami-032006127456fba8a\" ] }, \"region\": \"us-east-1\" } - ReconNum:1619647200000" 
| rex field=d "(?<JsonData>{[^}].+})" 
| spath input=JsonData output=myfield path=metrics.Data.DeletedImages 
| fields JsonData, myfield

[bitbucket]

A coworker and I actually just changed it to

| rex field=_raw "DeletedImages.:\s(?<DeletedImagesCount>\d+)"

Which works fine and I can get the result.  That's the only field I need so recasting as JSON doesn't really gain me anything.

Thanks for checking though!

[bitbucket]

Thanks for checking!  I confirmed that your run-anywhere example worked.

This is part of a larger search.  The full string  is:

index=aws_cloudwatch_log_default source="DOAF-logs/*" AND "Action completed" AND images-deleted | rex "(?<jsonData>{[^}].+})" | spath input=jsonData output=myfield path=metrics.Data.DeletedImages | fields jsonData, myfield

It does select the correct events, which are all similar to the example I posted earlier.  Yet when I go into table view and toggle jsonData and myfield to be displayed, both show NULL values.  Strangely, in the table view, it's also not pulling out source or sourceType - it's only getting the timestamp and _raw.

Edited to add - if I strip off the fields command I get source and sourcetype back.