Splunk Search

How to use makemv with tokenizers while keeping non-matching events?

Gunnar
Explorer

Hi,

I have events similar to this example:

1) date1, id1, misc

2) date2, id2, misc

3) date3, , misc

4) date4, id3 and id4, misc

The ids in 4) should be split into two separate lines.  The result should look like this:

1) date1, id1, misc

2) date2, id2, misc

3) date3, , misc

4) date4, id3 , misc

5) date4, id4, misc

But when using makemv with tokenizer lines which do not match, the tokenizers are skipped in the result, e.g.:

... | makemv tokenizer="(id\d)" ID | mvexpand ID | ...

Results in:

1) date1, id1, misc

2) date2, id2, misc

3) date4, id3 , misc

4) date4, id4, misc

How can I keep the non-matching lines? Is there a way to only use makemv where it is necessary?

My workaround at the moment is to append a second search that looks for events with an empty ID and adds those events again after the makemv, e,g,:

first search | ...  |  makemv tokenizer="(id\d)" ID | mvexpand ID | append [ first search again | ... | search NOT ID="*" | ... ] | ...

But searching twice can't be an optimal solution.

Thank you,

Gunnar

Labels (1)
Tags (2)
0 Karma
1 Solution

bowesmana
SplunkTrust
SplunkTrust

Not sure of the exact format of your data, but look at this example using split/mvexpand where fillnull is used to ensure the missing id gets preserved in the mvexpand

| makeresults count=4
| eval t=1 
| accum t
| eval _time=now() - (random() % 86400)
| eval id=case(t=1,"id1",t=2,"id2",t=3,null,t=4,"id3 id4")
| eval misc=random()
| fields - t
| eval id=split(id," ")
| fillnull value="" id
| mvexpand id

It's likely that fillnull will solve your problem in your example makemv case also - set the value to whatever makes sense in your context.

Hope this helps

 

View solution in original post

0 Karma

bowesmana
SplunkTrust
SplunkTrust

Not sure of the exact format of your data, but look at this example using split/mvexpand where fillnull is used to ensure the missing id gets preserved in the mvexpand

| makeresults count=4
| eval t=1 
| accum t
| eval _time=now() - (random() % 86400)
| eval id=case(t=1,"id1",t=2,"id2",t=3,null,t=4,"id3 id4")
| eval misc=random()
| fields - t
| eval id=split(id," ")
| fillnull value="" id
| mvexpand id

It's likely that fillnull will solve your problem in your example makemv case also - set the value to whatever makes sense in your context.

Hope this helps

 

0 Karma

Gunnar
Explorer

Hi bowesmana,

great idea - this put me on the right track.

What I do now is to use an eval before the makemv that creates an ID value if empty, that matches my tokenizer but cannot occur in the real data.

After the the mvexpand I use another eval to remove this dummy-ID again.

Seems to work fine. And two evals should be better than two searches.

Thank you very much and kind regards,

Gunnar

0 Karma
Get Updates on the Splunk Community!

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Announcing the General Availability of Splunk Enterprise Security 8.1!

We are pleased to announce the general availability of Splunk Enterprise Security 8.1. Splunk becomes the only ...

Developer Spotlight with William Searle

The Splunk Guy: A Developer’s Path from Web to Cloud William is a Splunk Professional Services Consultant with ...