Splunk Search

How to extract field with it's value enclosed by square bracket

Stan
New Member

When parsing some customized log, the format it's like below

[timestamps] field name [value]
[00:46:38]  - Remain Queue      [          0 ]
[00:46:38]  - Remain Queue      [          2 ]

The only search term works for me is "Remain Queue" NOT 0. I've tried "Remain Queue">0 but no luck. How can I search/sort the remain queue count? How to train splurk to know Remain Queue is a field?

Further more, I've got another log entry looks like below. Is it possible to parse it if I want key-value pair become Connect=93?

[00:46:38]  - Connect           [     330931 /     330838 ]     [         93 ]

Thanks.

Tags (2)
0 Karma
1 Solution

Johnvey
Contributor

Define your field extraction via a regex:

^\[[^\]]+\]\s+\-\s*([\w ]+).+\[\s*([^\s]+)\s*\]

This is the format that you would add to props.conf, where the matched groups $1 and $2 will extract the field data you want.

You can also try this inline via the rex command (as to avoid editing conf files):

... | rex "^\[[^\]]+\]\s+\-\s*(?<action>[\w ]+).+\[\s*(?<value>[^\s]+)\s*\]" | search action="Remain Queue" value>=10

where the named extractions will generate a action and value field. The subsequent search command then uses the newly extracted fields. This regex works on both examples you provided:

[00:46:38]  - Remain Queue      [          0 ]
[00:46:38]  - Remain Queue      [          2 ]
[00:46:38]  - Connect           [     330931 /     330838 ]     [         93 ]

You can test out the regexes online at a regex testing page.

View solution in original post

Johnvey
Contributor

Define your field extraction via a regex:

^\[[^\]]+\]\s+\-\s*([\w ]+).+\[\s*([^\s]+)\s*\]

This is the format that you would add to props.conf, where the matched groups $1 and $2 will extract the field data you want.

You can also try this inline via the rex command (as to avoid editing conf files):

... | rex "^\[[^\]]+\]\s+\-\s*(?<action>[\w ]+).+\[\s*(?<value>[^\s]+)\s*\]" | search action="Remain Queue" value>=10

where the named extractions will generate a action and value field. The subsequent search command then uses the newly extracted fields. This regex works on both examples you provided:

[00:46:38]  - Remain Queue      [          0 ]
[00:46:38]  - Remain Queue      [          2 ]
[00:46:38]  - Connect           [     330931 /     330838 ]     [         93 ]

You can test out the regexes online at a regex testing page.

Johnvey
Contributor

Ah, you need to add another search or where command after the rex because the field will only come into being after it, like: * | rex ... | where action="Remain Queue" value>10. I've updated the example.

0 Karma

Stan
New Member

The other question is there're so many props.conf files in splunk folder. Which one should I modify. And which section and what key should I add in?
eg.
[SECTION NAME]
KEY = ^[[^]]+]\s+-\s*([\w ]+).+[\s*([^\s]+)\s*]

0 Karma

Stan
New Member

Thanks for the awesome solution. But I am too new to get this work. I tried to search with '"Remain Queue" > 10 | rex "^[[^]]+]\s+-\s*(?[\w ]+).+[\s*(?[^\s]+)\s*]" but no luck. Even remove double quotes from Remain Queue still doesn't work. Why?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...