Getting Data In

How to parse all json fields in mixed unstructured+structured event?

zachsisinst
Explorer

Hi there, I have this type of event coming into splunk: 

```
[redacted:54407 24943076666] Processing MessageDispatcher.deliver_batch([#<Message::Queued:0x0000aaab14d8f418 @id=10440927, @created_at=Fri, 03 Jun 2022 14:21:43.890133282 UTC +00:00>, #<Message::Queued:0x0000aaab14dbc8c8 @id=10440928, @created_at=Fri, 03 Jun 2022 14:21:43.896693884 UTC +00:00>]{"
tag":"something","strand":null,"singleton":null,"priority":25,"attempts":0,"created_at":"2022-06-03T14:21:43Z","max_attempts":15,"source":"hostname:redacted,pid:29920"}
```

 

I would like to extract all of the json fields dynamically without individually pulling them out with multiple rex's. I have tried the following, but I am not seeing the json fields being parsed. `myjson` is successfully extracted, but spath does not pull out individual fields from the json: 

```
index="myindex" source="mysource" 
| rex field=_raw "(?<myjson>\{.+\})"
| spath myjson
```

Labels (2)
0 Karma
1 Solution

VatsalJagani
SplunkTrust
SplunkTrust

@zachsisinst - It should be:

| spath input=myjson

 

Here is my working search query:

| makeresults | eval _raw="[redacted:54407 24943076666] Processing MessageDispatcher.deliver_batch([#<Message::Queued:0x0000aaab14d8f418 @id=10440927, @created_at=Fri, 03 Jun 2022 14:21:43.890133282 UTC +00:00>, #<Message::Queued:0x0000aaab14dbc8c8 @id=10440928, @created_at=Fri, 03 Jun 2022 14:21:43.896693884 UTC +00:00>]{\"tag\":\"something\",\"strand\":null,\"singleton\":null,\"priority\":25,\"attempts\":0,\"created_at\":\"2022-06-03T14:21:43Z\",\"max_attempts\":15,\"source\":\"hostname:redacted,pid:29920\"}"
| rex field=_raw "(?<myjson>\{.+\})"
| spath input=myjson

 

I hope this helps!!!

View solution in original post

0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@zachsisinst - It should be:

| spath input=myjson

 

Here is my working search query:

| makeresults | eval _raw="[redacted:54407 24943076666] Processing MessageDispatcher.deliver_batch([#<Message::Queued:0x0000aaab14d8f418 @id=10440927, @created_at=Fri, 03 Jun 2022 14:21:43.890133282 UTC +00:00>, #<Message::Queued:0x0000aaab14dbc8c8 @id=10440928, @created_at=Fri, 03 Jun 2022 14:21:43.896693884 UTC +00:00>]{\"tag\":\"something\",\"strand\":null,\"singleton\":null,\"priority\":25,\"attempts\":0,\"created_at\":\"2022-06-03T14:21:43Z\",\"max_attempts\":15,\"source\":\"hostname:redacted,pid:29920\"}"
| rex field=_raw "(?<myjson>\{.+\})"
| spath input=myjson

 

I hope this helps!!!

0 Karma
Get Updates on the Splunk Community!

Leveraging Detections from the Splunk Threat Research Team & Cisco Talos

  Now On Demand  Stay ahead of today’s evolving threats with the combined power of the Splunk Threat Research ...

New in Splunk Observability Cloud: Automated Archiving for Unused Metrics

Automated Archival is a new capability within Metrics Management; which is a robust usage & cost optimization ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...