Getting Data In

Bulk rename fields created by spath in json search results

Cuyose
Builder

I am working with a bunch of different logs that contain json, sometimes for events that differ. I have the props set up so I can easily spath what I want. However even though the json is valid the structure is a bit messy and not consistent. Until we clear up the json logging, I wanted to be able to do my searches with what I have and bulk rename the fields returned. I can't find an easy way to do this that is, on-the-fly, customizable. Here is an example search and what I want to do.

I know I can do the following and get rid of the data.subObject{} string portion in auto-field extraction

index=main sourcetype=jsonEvents
|spath output=json path="data.subObject{}."
|spath input=json
|search *subObject* (this is to get specific events from the raw data)

But I would like to do this and just get rid of the data.subObject{} string in the field labels as there are many other json field extractions with different paths, but same subObject names.

index=main sourcetype=jsonEvents
|spath path="data.subObject{}."
|search *subObject* (this is to get specific events from the raw data)

Fields extracted here are :

data.subObject{}.a, data.subObject{}.b, data.subObject{}.c etc.

I want to rename them all to :

identifier.a, identifier.b, identifier.c etc.

0 Karma
1 Solution

somesoni2
Revered Legend

Try this

index=main sourcetype=jsonEvents |spath path="data.subObject{}." | rename "data.subObject{}."* as "identifier."*

View solution in original post

somesoni2
Revered Legend

Try this

index=main sourcetype=jsonEvents |spath path="data.subObject{}." | rename "data.subObject{}."* as "identifier."*

cmeo
Contributor

Bumping this topic: is there any way to do this in props? I keep finding json logs with preposterously long paths to the fields; it would be good to have a quick way to permanently clean this up in one spot.

0 Karma

Cuyose
Builder

This made my life easier in picking out relevant fields. Renaming the redundant fields to ZZZ, throws them at the end of the list, leaving only the extractions I targeted.

index=main sourcetype=jsonLog
|spath path="data.order."
|search NOT *string*
|rename "data.order.orderJSON.order_items{}."* as "items."*
|rename "data.order."* as "order."*
|rename "order.items{}."* as "ZZZ."*
|rename "order.orderJSON."* as "ZZZ."*

0 Karma

somesoni2
Revered Legend

The field extractions are sourcetype specific. I don't think its possible to save for a saved search. There is a work around that you can try though. For each saved search, you can create a search macro which basically execute 'rex' command to extract field and then use the macro in the saved search. This way only the fields extraction defined in a macro will be shown.

0 Karma

Cuyose
Builder

Thanks, this works for me! I was using rename after the fact. I have another question waiting to be answered on how I can get field extractions saved on a per-search basis.

The goal being so I don't have a list of hundreds of "selected fields" for all my different saved searches.

0 Karma
Get Updates on the Splunk Community!

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...

Security Highlights | January 2023 Newsletter

January 2023 Splunk Security Essentials (SSE) 3.7.0 ReleaseThe free Splunk Security Essentials (SSE) 3.7.0 app ...

Platform Highlights | January 2023 Newsletter

 January 2023Peace on Earth and Peace of Mind With Business ResilienceAll organizations can start the new year ...