Splunk Search

how can I use dedup command using many fields??

Explorer

Greetings!!

I would like to ask a question about dedup
eg: |dedup host ,IP
|dedup host |dedup IP
I've tried but when I use a comma, dedup works only on the first fields, and I want that this can be performed on both sides not only one side, I wanted that the output for fields 1 and fields 2 no redundancy values come again?

for example:
|dedup host, IP --->this brings me the below output: and I want that this one could not be repeated also like on host fields.

host IP

x 1.1.1.1
y 1.1.1.1
z 2.2.2.2

what the best way to remove redundancy for two fields????????
I need your help?

Thanks!

Tags (2)
0 Karma
1 Solution

Esteemed Legend

Like this:

...| dedup host
| dedup IP

View solution in original post

0 Karma

Explorer

Using dedup on multiple fields with the comma isn't only working on the first field. It is actually removing events where the host and IP BOTH match.

0 Karma

SplunkTrust
SplunkTrust
| makeresults 
| eval _raw="C IP
x 1.1.1.1
x 2.2.2.2
y 1.1.1.1
z 2.2.2.2"
| makemv delim="
" _raw
| multikv
| table C IP
| rename C as host
| dedup host
| dedup IP

OR

| makeresults 
| eval _raw="C IP
x 1.1.1.1
x 2.2.2.2
y 1.1.1.1
z 2.2.2.2"
| makemv delim="
" _raw
| multikv
| table C IP
| rename C as host
| stats count by host IP
| table host IP

OR

| makeresults 
| eval _raw="C IP
x 1.1.1.1
x 2.2.2.2
y 1.1.1.1
z 2.2.2.2"
| makemv delim="
" _raw
| multikv
| table C IP
| rename C as host
| dedup host,IP

Which result is correct?

0 Karma

Explorer

Thank you to4kawa and Woodcock,

Thank you for your assistance.

Esteemed Legend

Like this:

...| dedup host
| dedup IP

View solution in original post

0 Karma

SplunkTrust
SplunkTrust
| stats count by your_dedup

Hi, @pacifikn
this is the easy way.

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!