Dedup and multivalued fields

[jamesdon]

Hello,

I am extracting a few user names from a multiple line log, using MV_ADD=true in transforms.conf and KV_MODE = multi in transforms. The search results look great, and are what I am expecting; a single event per user found.

However, when I attempt to dedup the users, I run into problems. The user names are no longer individual events, but are consolidated into the original event. For instance, lets pretend that these are the three logs that I am searching:

---

Username: rabbit
Username: snail

---

Username: rabbit
Username: turtle

---

Username: rabbit

---

When I attempt to dedup the user name, I get:

---

rabbit, snail
rabbit, turtle
rabbit

---

I am expecting:

---

rabbit
snail
turtle

---

I thought that the problem would be resolved by using "multi: invokes multikv to expand a tabular event into multiple events" in props, but it does not.

Any ideas? I could always pre-process the data, but I was hoping to do it all in Splunk.

Thank you,

Jim

[Stephen_Sorkin · Accepted solution]

You have a couple choices here:

1. Use ... | mvexpand username to break each event into a distinct event per username, duplicating the message, followed by the dedup to find the most recent per username.
2. Use ... | stats count by username | fields username to just aggregate the list of username values.

The first is better if you want to see the original messages, but is less efficient than the second, which is better for just finding the user list.