Splunk Search
Highlighted

How to extract field with it's value enclosed by square bracket

New Member

When parsing some customized log, the format it's like below

[timestamps] field name [value]
[00:46:38]  - Remain Queue      [          0 ]
[00:46:38]  - Remain Queue      [          2 ]

The only search term works for me is "Remain Queue" NOT 0. I've tried "Remain Queue">0 but no luck. How can I search/sort the remain queue count? How to train splurk to know Remain Queue is a field?

Further more, I've got another log entry looks like below. Is it possible to parse it if I want key-value pair become Connect=93?

[00:46:38]  - Connect           [     330931 /     330838 ]     [         93 ]

Thanks.

Tags (2)
0 Karma
Highlighted

Re: How to extract field with it's value enclosed by square bracket

Contributor

Define your field extraction via a regex:

^\[[^\]]+\]\s+\-\s*([\w ]+).+\[\s*([^\s]+)\s*\]

This is the format that you would add to props.conf, where the matched groups $1 and $2 will extract the field data you want.

You can also try this inline via the rex command (as to avoid editing conf files):

... | rex "^\[[^\]]+\]\s+\-\s*(?<action>[\w ]+).+\[\s*(?<value>[^\s]+)\s*\]" | search action="Remain Queue" value>=10

where the named extractions will generate a action and value field. The subsequent search command then uses the newly extracted fields. This regex works on both examples you provided:

[00:46:38]  - Remain Queue      [          0 ]
[00:46:38]  - Remain Queue      [          2 ]
[00:46:38]  - Connect           [     330931 /     330838 ]     [         93 ]

You can test out the regexes online at a regex testing page.

View solution in original post

Highlighted

Re: How to extract field with it's value enclosed by square bracket

New Member

Thanks for the awesome solution. But I am too new to get this work. I tried to search with '"Remain Queue" > 10 | rex "^[[^]]+]\s+-\s(?[\w ]+).+[\s(?[^\s]+)\s*]" but no luck. Even remove double quotes from Remain Queue still doesn't work. Why?

0 Karma
Highlighted

Re: How to extract field with it's value enclosed by square bracket

New Member

The other question is there're so many props.conf files in splunk folder. Which one should I modify. And which section and what key should I add in?
eg.
[SECTION NAME]
KEY = ^[[^]]+]\s+-\s([\w ]+).+[\s([^\s]+)\s*]

0 Karma
Highlighted

Re: How to extract field with it's value enclosed by square bracket

Contributor

Ah, you need to add another search or where command after the rex because the field will only come into being after it, like: * | rex ... | where action="Remain Queue" value>10. I've updated the example.

0 Karma