Splunk Search

how can I use dedup command using many fields??

pacifikn
Communicator

Greetings!!

I would like to ask a question about dedup
eg: |dedup host ,IP
|dedup host |dedup IP
I've tried but when I use a comma, dedup works only on the first fields, and I want that this can be performed on both sides not only one side, I wanted that the output for fields 1 and fields 2 no redundancy values come again?

for example:
|dedup host, IP --->this brings me the below output: and I want that this one could not be repeated also like on host fields.

host IP

x 1.1.1.1
y 1.1.1.1
z 2.2.2.2

what the best way to remove redundancy for two fields????????
I need your help?

Thanks!

Tags (2)
0 Karma
1 Solution

woodcock
Esteemed Legend

Like this:

...| dedup host
| dedup IP

View solution in original post

0 Karma

bjcross
Explorer

Using dedup on multiple fields with the comma isn't only working on the first field. It is actually removing events where the host and IP BOTH match.

0 Karma

to4kawa
Ultra Champion
| makeresults 
| eval _raw="C IP
x 1.1.1.1
x 2.2.2.2
y 1.1.1.1
z 2.2.2.2"
| makemv delim="
" _raw
| multikv
| table C IP
| rename C as host
| dedup host
| dedup IP

OR

| makeresults 
| eval _raw="C IP
x 1.1.1.1
x 2.2.2.2
y 1.1.1.1
z 2.2.2.2"
| makemv delim="
" _raw
| multikv
| table C IP
| rename C as host
| stats count by host IP
| table host IP

OR

| makeresults 
| eval _raw="C IP
x 1.1.1.1
x 2.2.2.2
y 1.1.1.1
z 2.2.2.2"
| makemv delim="
" _raw
| multikv
| table C IP
| rename C as host
| dedup host,IP

Which result is correct?

0 Karma

pacifikn
Communicator

Thank you to4kawa and Woodcock,

Thank you for your assistance.

woodcock
Esteemed Legend

Like this:

...| dedup host
| dedup IP
0 Karma

to4kawa
Ultra Champion
| stats count by your_dedup

Hi, @pacifikn
this is the easy way.

0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...