Splunk Search
Highlighted

Dedup and multivalued fields

Path Finder

Hello,

I am extracting a few user names from a multiple line log, using MV_ADD=true in transforms.conf and KV_MODE = multi in transforms. The search results look great, and are what I am expecting; a single event per user found.

However, when I attempt to dedup the users, I run into problems. The user names are no longer individual events, but are consolidated into the original event. For instance, lets pretend that these are the three logs that I am searching:


Username: rabbit
Username: snail


Username: rabbit
Username: turtle


Username: rabbit


When I attempt to dedup the user name, I get:


rabbit, snail
rabbit, turtle
rabbit


I am expecting:


rabbit
snail
turtle


I thought that the problem would be resolved by using "multi: invokes multikv to expand a tabular event into multiple events" in props, but it does not.

Any ideas? I could always pre-process the data, but I was hoping to do it all in Splunk.

Thank you,

Jim

Tags (2)
0 Karma
Highlighted

Re: Dedup and multivalued fields

Splunk Employee
Splunk Employee

You have a couple choices here:

  1. Use ... | mvexpand username to break each event into a distinct event per username, duplicating the message, followed by the dedup to find the most recent per username.
  2. Use ... | stats count by username | fields username to just aggregate the list of username values.

The first is better if you want to see the original messages, but is less efficient than the second, which is better for just finding the user list.

View solution in original post