Getting Data In

Bulk rename fields created by spath in json search results

Cuyose
Builder

I am working with a bunch of different logs that contain json, sometimes for events that differ. I have the props set up so I can easily spath what I want. However even though the json is valid the structure is a bit messy and not consistent. Until we clear up the json logging, I wanted to be able to do my searches with what I have and bulk rename the fields returned. I can't find an easy way to do this that is, on-the-fly, customizable. Here is an example search and what I want to do.

I know I can do the following and get rid of the data.subObject{} string portion in auto-field extraction

index=main sourcetype=jsonEvents
|spath output=json path="data.subObject{}."
|spath input=json
|search *subObject* (this is to get specific events from the raw data)

But I would like to do this and just get rid of the data.subObject{} string in the field labels as there are many other json field extractions with different paths, but same subObject names.

index=main sourcetype=jsonEvents
|spath path="data.subObject{}."
|search *subObject* (this is to get specific events from the raw data)

Fields extracted here are :

data.subObject{}.a, data.subObject{}.b, data.subObject{}.c etc.

I want to rename them all to :

identifier.a, identifier.b, identifier.c etc.

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

Try this

index=main sourcetype=jsonEvents |spath path="data.subObject{}." | rename "data.subObject{}."* as "identifier."*

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

Try this

index=main sourcetype=jsonEvents |spath path="data.subObject{}." | rename "data.subObject{}."* as "identifier."*

cmeo
Contributor

Bumping this topic: is there any way to do this in props? I keep finding json logs with preposterously long paths to the fields; it would be good to have a quick way to permanently clean this up in one spot.

0 Karma

Cuyose
Builder

This made my life easier in picking out relevant fields. Renaming the redundant fields to ZZZ, throws them at the end of the list, leaving only the extractions I targeted.

index=main sourcetype=jsonLog
|spath path="data.order."
|search NOT *string*
|rename "data.order.orderJSON.order_items{}."* as "items."*
|rename "data.order."* as "order."*
|rename "order.items{}."* as "ZZZ."*
|rename "order.orderJSON."* as "ZZZ."*

0 Karma

somesoni2
SplunkTrust
SplunkTrust

The field extractions are sourcetype specific. I don't think its possible to save for a saved search. There is a work around that you can try though. For each saved search, you can create a search macro which basically execute 'rex' command to extract field and then use the macro in the saved search. This way only the fields extraction defined in a macro will be shown.

0 Karma

Cuyose
Builder

Thanks, this works for me! I was using rename after the fact. I have another question waiting to be answered on how I can get field extractions saved on a per-search basis.

The goal being so I don't have a list of hundreds of "selected fields" for all my different saved searches.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...