topic Re: Rename using wildcard in Getting Data In

Rename using wildcard

tbo — Tue, 07 Jan 2014 12:21:59 GMT

I get some json-formatted logs, that I want to extract a field from. It looks something like the following:

{
    "ack-count": {
        "spout:__ack_init": 12,
        "bolt-one:__ack_ack": 6,
        "bolt-two:__ack_ack": 2,
        "bolt-three:__ack_ack": 4,
        "error-bolt:__ack_ack": 12
    }
}

I need to fetch the value of the field ack-count.spout:__ack_init, here 12, and use it in some calculation. My problem is that I don't know the exact name a priori. I only know that it ends with __ack_init, and lies within the field ack-count. That is, it can be matched with ack-count*__ack_init.

So, what I wanted to do was something like

... | rename ack-count*__ack_init AS acks

But when I try that, I get the error

Error in 'rename' command: Wildcard mismatch: 'ack-count*__ack_init' as 'acks'.

Can anyone tell me how that might be?

By the way, there might be more than one field like that, in which case I would like to sum them, before using them...

Re: Rename using wildcard

alacercogitatus — Tue, 07 Jan 2014 13:38:49 GMT

Try doing it this way:

your_search | spath path=ack-count output=a_tmp | rex field=a_tmp "[^:]*:__ack_init\":\s(?<acks>\d)"

Spath takes your json and makes it a string, with which you can do a rex on.

Re: Rename using wildcard

tbo — Tue, 07 Jan 2014 14:13:36 GMT

Thanks. But you have that word "spout" as part of your regex. And I can't be sure that will always be present. The field might be called foo:__ack_init, in which case I suppose the search will fail?

Re: Rename using wildcard

somesoni2 — Mon, 28 Sep 2020 15:36:04 GMT

try changing rex command above with this.

| rex field=a_tmp "\"[^a-zA-Z_:]:\s\"?(?\d*)"

Re: Rename using wildcard

alacercogitatus — Tue, 07 Jan 2014 17:23:42 GMT

heh, beat me too it. But I would use this:

rex field=a_tmp "[^:]*:__ack_init":\s(?<acks>\d)"

To avoid picking up the other acks.

Re: Rename using wildcard

tbo — Tue, 07 Jan 2014 19:03:51 GMT

Thanks man! Could you update your answer? Then I'll accept it.

Re: Rename using wildcard

ktwingstrom — Wed, 17 Oct 2018 17:10:53 GMT

Is there a way to accomplish this same principle in a props file? I've got JSON data that has a return{} field that has a big list of software that needs to be updated. But splunk is auto extracting each software name out as part of the field name. I want to do the same thing the OP did and rename all fields called Return* as something else. Can this be done in a conf file? Thanks