How to extract field with it's value enclosed by square bracket

[Stan]

When parsing some customized log, the format it's like below

[timestamps] field name [value]
[00:46:38]  - Remain Queue      [          0 ]
[00:46:38]  - Remain Queue      [          2 ]

The only search term works for me is "Remain Queue" NOT 0. I've tried "Remain Queue">0 but no luck. How can I search/sort the remain queue count? How to train splurk to know Remain Queue is a field?

Further more, I've got another log entry looks like below. Is it possible to parse it if I want key-value pair become Connect=93?

[00:46:38]  - Connect           [     330931 /     330838 ]     [         93 ]

Thanks.

[Johnvey · Accepted solution]

Define your field extraction via a regex:

^\[[^\]]+\]\s+\-\s*([\w ]+).+\[\s*([^\s]+)\s*\]

This is the format that you would add to props.conf, where the matched groups $1 and $2 will extract the field data you want.

You can also try this inline via the rex command (as to avoid editing conf files):

... | rex "^\[[^\]]+\]\s+\-\s*(?<action>[\w ]+).+\[\s*(?<value>[^\s]+)\s*\]" | search action="Remain Queue" value>=10

where the named extractions will generate a action and value field. The subsequent search command then uses the newly extracted fields. This regex works on both examples you provided:

[00:46:38]  - Remain Queue      [          0 ]
[00:46:38]  - Remain Queue      [          2 ]
[00:46:38]  - Connect           [     330931 /     330838 ]     [         93 ]

You can test out the regexes online at a regex testing page.

[Johnvey]

Ah, you need to add another search or where command after the rex because the field will only come into being after it, like: * | rex ... | where action="Remain Queue" value>10. I've updated the example.

[Stan]

The other question is there're so many props.conf files in splunk folder. Which one should I modify. And which section and what key should I add in?
eg.
[SECTION NAME]
KEY = ^[[^]]+]\s+-\s*([\w ]+).+[\s*([^\s]+)\s*]

[Stan]

Thanks for the awesome solution. But I am too new to get this work. I tried to search with '"Remain Queue" > 10 | rex "^[[^]]+]\s+-\s*(?[\w ]+).+[\s*(?[^\s]+)\s*]" but no luck. Even remove double quotes from Remain Queue still doesn't work. Why?