Splunk Search

Bulk rename of all fields that come from calling "| lookup"?

woodcock
Esteemed Legend

We have a CMDB lookup that adds 100 fields when we do not specify a limited set with "OUTPUT". More fields are added every now and then. I cannot rename the fields in the file because too many people share this file; it is what it is. Is there a flexible way to rename all fields that are in there that will still work even when new fields are added or old field names change slightly? If I jus.t do an iteration of every field with | rename a AS CMDB_a, | rename b AS CMDB_b ... this will be fragile and break and be a huge blight in the middle of my search. I cannot do | rename * AS CMDB_* because there are many other fields and this search might change to add more non-CMDB fields, too.

1 Solution

Noah_Woodcock
Path Finder

Like this:

... | rename * AS _HIDE_ALL_FIELDS_*
| lookup YourLookupDefinitionHere host AS _HIDE_ALL_FIELDS_host
| rename * AS CMDB_*
| rename _HIDE_ALL_FIELDS_* AS *

View solution in original post

Noah_Woodcock
Path Finder

Like this:

... | lookup YourLookupDefinitionHere host
| rename COMMENT AS "Bulk rename of all fields from this lookup to add prefix 'CMDB_'"
| rename [| inputlookup  YourLookupDefinitionHere
          | head 1
          | foreach * [ eval <<FIELD>> = "CMDB_<<FIELD>>" ]
          | format "" "" "" "" "" ""
| rex field=search mode=sed "s/=/ AS /g"]

Noah_Woodcock
Path Finder

Like this:

... | rename * AS _HIDE_ALL_FIELDS_*
| lookup YourLookupDefinitionHere host AS _HIDE_ALL_FIELDS_host
| rename * AS CMDB_*
| rename _HIDE_ALL_FIELDS_* AS *

theady
Engager

I had issue with this. it seems to take the _* fields as part of the * on the first rename.

0 Karma

woodcock
Esteemed Legend

It definitely should not and is a bug if it does.

0 Karma

bowesmana
SplunkTrust
SplunkTrust

Came across this solution and noted the comments about _ fields being renamed and it does appear that it won't work as _ fields are also renamed with *

| makeresults
| eval myField=1
| rename * as HIDE_*
| table *

shows HIDE__time

and in my example

| rename * as _HIDE_ALL_*
| lookup work_status key as _HIDE_ALL_workStatusKey
| rename * as orig_work_status_*
| rename _HIDE_ALL_* as *

 after the second rename, all _HIDE_ fields are now orig_work_status__HIDE_...

which is not the intended outcome - shame - as it was a simple approach.

 

0 Karma

davewood
Explorer

It seems the rename command changed behaviour somewhere along the way and now, as you noted, * matches _ fields too.

You can still use the same method though with a tweak to the final rename:

| rename * as _HIDE_ALL_*
| lookup work_status key as _HIDE_ALL_workStatusKey
| rename * as orig_work_status_*
| rename orig_work_status__HIDE_ALL_* as *

 

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...