Splunk Search

Splunk Create Fields from Field Values in Json log

ssjabid
Explorer

Hi, 

I am trying to create new field values from my json log base on the values that appear under a particular field

So here is an example

{
   "widget": {
       "text": [ 
        {
           "data": "Click here",
           "size": 36
        },
       {
          "data": "Learn more",
          "size": 37
       },
       {
          "data": "Help",
          "size": 38
       },
       ]
   }
}

 So in my environment I currently have got widget{}.text{}.data as a field, however i would like to break it further and have

widget{}.text{}.data{}.ClickHere, 

widget{}.text{}.data{}.Help, 

widget{}.text{}.data{}.LearnMore as individual fields

I ask this because when we have thousands of logs and are looking for certain combinations, we have issues with filtering accurately, doing this will help us find the right combinations 

Any assistance will be greatly appreciated,

Thanks 

Labels (6)
Tags (1)
0 Karma
1 Solution

to4kawa
Ultra Champion

 

| makeresults
| eval _raw="{
   \"widget\": {
       \"text\": [ 
        {
           \"data\": \"Click here\",
           \"size\": 36
        },
       {
          \"data\": \"Learn more\",
          \"size\": 37
       },
       {
          \"data\": \"Help\",
          \"size\": 38
       },
       ]
   }
}"
| rex max_match=0 "(?ms)(?<jsons>{\s*\"data.*?})"
| stats count by jsons
| spath input=jsons
| eval {data}=size
| table "Click here" Help "Learn more"
| stats values(*) as *

 

View solution in original post

to4kawa
Ultra Champion

 

| makeresults
| eval _raw="{
   \"widget\": {
       \"text\": [ 
        {
           \"data\": \"Click here\",
           \"size\": 36
        },
       {
          \"data\": \"Learn more\",
          \"size\": 37
       },
       {
          \"data\": \"Help\",
          \"size\": 38
       },
       ]
   }
}"
| rex max_match=0 "(?ms)(?<jsons>{\s*\"data.*?})"
| stats count by jsons
| spath input=jsons
| eval {data}=size
| table "Click here" Help "Learn more"
| stats values(*) as *

 

ssjabid
Explorer

Thank you for this, i will try it out on monday, it looks like it will work, 

I've heard online about {} creating a new set of fields from field values

what is the purpose of using the rex? is that just to capture data into a new field so that we can use spath on it? is this part absolutely necessary?

if you could explain exactly what {data}=size does that will help me understand its real use for further scenarios, 

Thanks again, looking forward to trying this out!

0 Karma

to4kawa
Ultra Champion
| makeresults
| eval _raw="{
   \"widget\": {
       \"text\": [ 
        {
           \"data\": \"Click here\",
           \"size\": 36
        },
       {
          \"data\": \"Learn more\",
          \"size\": 37
       },
       {
          \"data\": \"Help\",
          \"size\": 38
       },
       ]
   }
}"
| spath
| rename widget.text{}.* as *
| eval tmp=mvzip(data,size)
| stats count by tmp
| eval data=mvindex(split(tmp,","),0), size=mvindex(split(tmp,","),1)
| xyseries count data size
| fields - count

 

1. I like rex to extract json array.(it's better to use mvzip,I guess)

2. see https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Eval#Field_names

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Quick connection discovery mode for forwarders

When a Splunk forwarder loses connectivity to its indexers, it does not always reconnect immediately. In many ...

Build and Launch AI Agents from Your Splunk Workflows

  Register We’ve all been there: juggling alerts, runbooks, and endless manual searches. What if you could ...

Splunk Cloud Application Management in Terraform

Register   On Tuesday, August 4 at 11AM PDT / 2PM EDT, we’re diving into how you can bring Infrastructure as ...