How to rename dynamic fields ?

[karthi2809]

Hi All,

Thanks in Advance

I have a requirement we are onboarding CSV files that contain events. I am writing query to display in the data in a table, and these events contain some dynamic fields names. Since the field names are lengthy a I need to shorten them for better readability. I tried using rename command in Splunk, but it did not work as expected

Could you please help me with how to rename multiple dynamic fields to shorter names?

Index=Test sourceType=csv  | fillnull value="NA"|rename "count/deploymentconfigs.apps.os.io_max_configured" AS "max_configured"|table index host sourcetype *|fields - _raw,_time

[ddrillic]

I would probably create a python script that does the renaming and sanity checks on the incoming csv file(s).

[ITWhisperer]

Firstly, you should consider renaming the fields in the csv file before onboarding, this will make your life easier going forward.

Assuming you can't/won't do this, do you have a name conversion scheme or a few schemes that can be applied automatically e.g. last word, or last two words?

Here is a method for taking the last word

| appendpipe
    [
    ``` Transpose to get names of fields ```
    | transpose 1 column_name=fields
    ``` remove row 1 ```
    | fields - "row 1"
    ``` Extract the new name from the current name ```
    | rex field=fields ".*?_(?<new_name>[a-zA-Z]+$)"
    ``` Rename all the fields so they are easy to identify ```
    | eval fields="R_".fields
    ``` Transpose to get renaming fields with their new names ```
    | transpose 0 header_field=fields
    ``` Remove column names ```
    | fields - column
    ]
``` Copy rename fields to all events ```
| reverse
| filldown R_*
| reverse
``` Remove appended event - this relies on knowing an original field which is non-null in all events ```
| where isnotnull(XYZ_Surname)
``` For each renaming field ```
| foreach R_*
    [
    ``` Use contents of renaming field to create renamed field with contents of original field ```
    | eval {<<FIELD>>}=<<MATCHSEG1>>
    ``` Remove original field and renaming field ```
    | fields - <<MATCHSEG1>> <<FIELD>>
    ]

You can modify this to use your own renaming schemes. You may also need to take into account any special, non-alphanumeric characters which appear in your field names by using single and double quotes in appropriate places.

[PickleRick]

Aren't you overcomplicating this? 😉

| foreach *
[
``` Theoretically you should make exceptions for the metadata fields but I'm too lazy 😉 ```
``` Here we're stripping first letter from each field name; adjust to your own needs ```
| eval mynewfieldname=replace("<<FIELD>>","^.","")
| eval {mynewfieldname}='<<FIELD>>'
| eval '<<FIELD>>'=null()
| eval mynewfieldname=null()
]

Big fat warning - can cause issues if the field names collide. Both resulting fields as well as temporary fields (as if you wanted to rename field1, field2, field3... to field2, field3, field4 respectively and started at the wrong end). 

[PickleRick]

1. What do you mean by "did not work as expected"? What does your data look like?

2. Using fields after table makes  no sense.