Splunk Search

How to dedup by field A for a subset of data, and dedup by field B on another subset?

sugethakch
New Member

In my whole data set, say, I have 3 types of data: Fan, Power and Transceiver.
On the Fan and Power, the unique field is 'Slot', but on the transceiver 'interface' is the unique field.

If I use the dedup on 'slot' field, then the transceiver data doesn't appear because the 'Slot' field is empty. The Fan and Power appear fine. But I want the transceiver to be have duplicates removed on another field, 'interface' and the Fans and Power on the field 'Slot'.

How do I do that?

Tags (3)
0 Karma

somesoni2
Revered Legend

Try this

your base search like sourcetype=Fan OR sourcetype=Power OR sourcetype=Transceiver | eval commonField=coalesce(Slot,interface) | dedup commonField

sugethakch
New Member

@somesoni2: But that would give me a new field called commonField. But I want both the fields to be displayed only the duplicates removed on 'slot' for Fans and Power and on 'interface' for Transceivers.

I want the info to display like this:

Matching events
name slot make model interface description serialnum hwrev mfgdate
A 4 FAN-1-F N/A

AB 3 FAN-1-F N/A

AC 2 FAN-1-F N/A

AD 1 FAN-1-F N/A

B P1 PWR-1-F N/A

BB P2 PWR-42-F N/A

C T1 INTF1 N/A 01.01 2012-02-29
CB T2 INTF2 N/A 01.01 2012-02-29

as you can see, there are no duplicates on the fans and power if i do a dedup on 'slot', but the transceiver data would be lost. On the other hand, if I do a dedup on 'interface', then the fans and power info will be lost.

0 Karma
Get Updates on the Splunk Community!

Detecting Brute Force Account Takeover Fraud with Splunk

This article is the second in a three-part series exploring advanced fraud detection techniques using Splunk. ...

Buttercup Games: Further Dashboarding Techniques (Part 9)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...

Buttercup Games: Further Dashboarding Techniques (Part 8)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...