topic Re: Bulk rename fields created by spath in json search results in Getting Data In

Bulk rename fields created by spath in json search results

Cuyose — Fri, 30 May 2014 17:34:08 GMT

I am working with a bunch of different logs that contain json, sometimes for events that differ. I have the props set up so I can easily spath what I want. However even though the json is valid the structure is a bit messy and not consistent. Until we clear up the json logging, I wanted to be able to do my searches with what I have and bulk rename the fields returned. I can't find an easy way to do this that is, on-the-fly, customizable. Here is an example search and what I want to do.

I know I can do the following and get rid of the data.subObject{} string portion in auto-field extraction

index=main sourcetype=jsonEvents
|spath output=json path="data.subObject{}."
|spath input=json
|search *subObject* (this is to get specific events from the raw data)

But I would like to do this and just get rid of the data.subObject{} string in the field labels as there are many other json field extractions with different paths, but same subObject names.

index=main sourcetype=jsonEvents
|spath path="data.subObject{}."
|search *subObject* (this is to get specific events from the raw data)

Fields extracted here are :

data.subObject{}.a, data.subObject{}.b, data.subObject{}.c etc.

I want to rename them all to :

identifier.a, identifier.b, identifier.c etc.

Re: Bulk rename fields created by spath in json search results

somesoni2 — Fri, 30 May 2014 17:48:33 GMT

Try this

index=main sourcetype=jsonEvents |spath path="data.subObject{}." | rename "data.subObject{}."* as "identifier."*

Re: Bulk rename fields created by spath in json search results

Cuyose — Fri, 30 May 2014 18:04:03 GMT

Thanks, this works for me! I was using rename after the fact. I have another question waiting to be answered on how I can get field extractions saved on a per-search basis.

The goal being so I don't have a list of hundreds of "selected fields" for all my different saved searches.

Re: Bulk rename fields created by spath in json search results

somesoni2 — Fri, 30 May 2014 18:29:15 GMT

The field extractions are sourcetype specific. I don't think its possible to save for a saved search. There is a work around that you can try though. For each saved search, you can create a search macro which basically execute 'rex' command to extract field and then use the macro in the saved search. This way only the fields extraction defined in a macro will be shown.

Re: Bulk rename fields created by spath in json search results

Cuyose — Fri, 30 May 2014 18:56:19 GMT

This made my life easier in picking out relevant fields. Renaming the redundant fields to ZZZ, throws them at the end of the list, leaving only the extractions I targeted.

index=main sourcetype=jsonLog
|spath path="data.order."
|search NOT *string*
|rename "data.order.orderJSON.order_items{}."* as "items."*
|rename "data.order."* as "order."*
|rename "order.items{}."* as "ZZZ."*
|rename "order.orderJSON."* as "ZZZ."*

Re: Bulk rename fields created by spath in json search results

cmeo — Sun, 29 Nov 2015 03:30:33 GMT

Bumping this topic: is there any way to do this in props? I keep finding json logs with preposterously long paths to the fields; it would be good to have a quick way to permanently clean this up in one spot.