Getting Data In

How to parse all json fields in mixed unstructured+structured event?

zachsisinst
Explorer

Hi there, I have this type of event coming into splunk: 

```
[redacted:54407 24943076666] Processing MessageDispatcher.deliver_batch([#<Message::Queued:0x0000aaab14d8f418 @id=10440927, @created_at=Fri, 03 Jun 2022 14:21:43.890133282 UTC +00:00>, #<Message::Queued:0x0000aaab14dbc8c8 @id=10440928, @created_at=Fri, 03 Jun 2022 14:21:43.896693884 UTC +00:00>]{"
tag":"something","strand":null,"singleton":null,"priority":25,"attempts":0,"created_at":"2022-06-03T14:21:43Z","max_attempts":15,"source":"hostname:redacted,pid:29920"}
```

 

I would like to extract all of the json fields dynamically without individually pulling them out with multiple rex's. I have tried the following, but I am not seeing the json fields being parsed. `myjson` is successfully extracted, but spath does not pull out individual fields from the json: 

```
index="myindex" source="mysource" 
| rex field=_raw "(?<myjson>\{.+\})"
| spath myjson
```

Labels (2)
0 Karma
1 Solution

VatsalJagani
SplunkTrust
SplunkTrust

@zachsisinst - It should be:

| spath input=myjson

 

Here is my working search query:

| makeresults | eval _raw="[redacted:54407 24943076666] Processing MessageDispatcher.deliver_batch([#<Message::Queued:0x0000aaab14d8f418 @id=10440927, @created_at=Fri, 03 Jun 2022 14:21:43.890133282 UTC +00:00>, #<Message::Queued:0x0000aaab14dbc8c8 @id=10440928, @created_at=Fri, 03 Jun 2022 14:21:43.896693884 UTC +00:00>]{\"tag\":\"something\",\"strand\":null,\"singleton\":null,\"priority\":25,\"attempts\":0,\"created_at\":\"2022-06-03T14:21:43Z\",\"max_attempts\":15,\"source\":\"hostname:redacted,pid:29920\"}"
| rex field=_raw "(?<myjson>\{.+\})"
| spath input=myjson

 

I hope this helps!!!

View solution in original post

0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@zachsisinst - It should be:

| spath input=myjson

 

Here is my working search query:

| makeresults | eval _raw="[redacted:54407 24943076666] Processing MessageDispatcher.deliver_batch([#<Message::Queued:0x0000aaab14d8f418 @id=10440927, @created_at=Fri, 03 Jun 2022 14:21:43.890133282 UTC +00:00>, #<Message::Queued:0x0000aaab14dbc8c8 @id=10440928, @created_at=Fri, 03 Jun 2022 14:21:43.896693884 UTC +00:00>]{\"tag\":\"something\",\"strand\":null,\"singleton\":null,\"priority\":25,\"attempts\":0,\"created_at\":\"2022-06-03T14:21:43Z\",\"max_attempts\":15,\"source\":\"hostname:redacted,pid:29920\"}"
| rex field=_raw "(?<myjson>\{.+\})"
| spath input=myjson

 

I hope this helps!!!

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Persistent Queue at TcpOut — One of Splunk's Most Practical Features

Splunk introduced persistent queueing at the tcpout layer as one of the most practical resilience features in ...

Build resilience for the agentic era: Join us at Splunk Go San Jose

AI is changing how organizations build, operate, and secure digital services. As teams adopt AI agents and ...

Manage Edge Processor pipelines end to end with APIs

This guide uses the Data Management APIs to create an Edge Processor, onboard one instance, deploy a pipeline ...