topic Re: How do I extract and separate an arbitrary number of field values with regex? in Splunk Search

How do I extract and separate an arbitrary number of field values with regex?

andrew207 — Tue, 28 Jul 2015 03:24:08 GMT

input:

myCommand -myArgs taska taskb taskc
myCommand -myArgs taska
myCommand -myArgs taska taskb taskc taskd

What's the best way to capture this? At the moment I'm using the regex

myCommand (?P<args>\-\w+)(\s(?P<tasks>[A-z0-9\s]+))

It results in

1. tasks: "taska taskb taskc"
2. tasks: "taska"
3. tasks: "taska taskb taskc taskd"

How would I go about separating these or making them individual? I want to aggregate by "taska" and draw some nice graphs more easily.

Re: How do I extract and separate an arbitrary number of field values with regex?

dineshraj9 — Tue, 28 Jul 2015 06:18:49 GMT

If you have 4 tasks, you can extract them separately in 4 columns -

(?P<args>\-\w+)\s+(?P<taskA>\w+)\s+(?P<taskB>\w+)\s+(?P<taskC>\w+)\s+(?P<taskD>\w+)

Re: How do I extract and separate an arbitrary number of field values with regex?

jeffland — Tue, 28 Jul 2015 06:48:35 GMT

I don't think this is possible with regex in the sense of arbitrary number of capturing groups. You could of course define a high number of capturing groups and make them optional, something like

(?<task1>\w+)(?:[ ])?(?<task2>\w+)?(?:[ ])?(?<task3>\w+)?(?:[ ])?(?<task4>\w+)?(?:[ ])?(?<task5>\w+)?

It shouldn't matter if the latter groups don't match.

But do you really need all those tasks as unique fields? What about the order they appear in, will that be fixed? Or, what if one event has task1=ab task2=cd and another event has task1=cd? You won't directly see that these two events both have a task "cd", but it is up to you to judge your needs.

I would suggest you keep them as one field the way you have them now, and do makemv tasks to have them in a multivalue field. That will allow you to compare them more easily and check which events contain which tasks.

Re: How do I extract and separate an arbitrary number of field values with regex?

andrew207 — Wed, 29 Jul 2015 02:43:22 GMT

makemv suited my purposes.

An arbitrary number of fields should be scalable. Your solution would be a bit messy if there were thousands of "tasks". Thanks though!

Re: How do I extract and separate an arbitrary number of field values with regex?

jeffland — Wed, 29 Jul 2015 07:43:46 GMT

And that's where regular expressions hit their limit - they don't support having an arbitrary number of capturing groups (not as far as I know, and quick googling reveals this evidence).