Getting Data In

Bulk rename fields created by spath in json search results

Builder

I am working with a bunch of different logs that contain json, sometimes for events that differ. I have the props set up so I can easily spath what I want. However even though the json is valid the structure is a bit messy and not consistent. Until we clear up the json logging, I wanted to be able to do my searches with what I have and bulk rename the fields returned. I can't find an easy way to do this that is, on-the-fly, customizable. Here is an example search and what I want to do.

I know I can do the following and get rid of the data.subObject{} string portion in auto-field extraction

index=main sourcetype=jsonEvents
|spath output=json path="data.subObject{}."
|spath input=json
|search *subObject* (this is to get specific events from the raw data)

But I would like to do this and just get rid of the data.subObject{} string in the field labels as there are many other json field extractions with different paths, but same subObject names.

index=main sourcetype=jsonEvents
|spath path="data.subObject{}."
|search *subObject* (this is to get specific events from the raw data)

Fields extracted here are :

data.subObject{}.a, data.subObject{}.b, data.subObject{}.c etc.

I want to rename them all to :

identifier.a, identifier.b, identifier.c etc.

0 Karma
1 Solution

Revered Legend

Try this

index=main sourcetype=jsonEvents |spath path="data.subObject{}." | rename "data.subObject{}."* as "identifier."*

View solution in original post

Revered Legend

Try this

index=main sourcetype=jsonEvents |spath path="data.subObject{}." | rename "data.subObject{}."* as "identifier."*

View solution in original post

Contributor

Bumping this topic: is there any way to do this in props? I keep finding json logs with preposterously long paths to the fields; it would be good to have a quick way to permanently clean this up in one spot.

0 Karma

Builder

This made my life easier in picking out relevant fields. Renaming the redundant fields to ZZZ, throws them at the end of the list, leaving only the extractions I targeted.

index=main sourcetype=jsonLog
|spath path="data.order."
|search NOT *string*
|rename "data.order.orderJSON.order_items{}."* as "items."*
|rename "data.order."* as "order."*
|rename "order.items{}."* as "ZZZ."*
|rename "order.orderJSON."* as "ZZZ."*

0 Karma

Revered Legend

The field extractions are sourcetype specific. I don't think its possible to save for a saved search. There is a work around that you can try though. For each saved search, you can create a search macro which basically execute 'rex' command to extract field and then use the macro in the saved search. This way only the fields extraction defined in a macro will be shown.

0 Karma

Builder

Thanks, this works for me! I was using rename after the fact. I have another question waiting to be answered on how I can get field extractions saved on a per-search basis.

The goal being so I don't have a list of hundreds of "selected fields" for all my different saved searches.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!