Why is dedup not working with this search?

[ErraticIncome93]

When I run the following search, I get 100+ results of src_ip 1.2.3.4 and signature X:

index=http status=200 src_ip!=10.0.0.0/8 src_ip!=192.168.0.0/16 src_ip!=172.16.0.0/12 | table src_ip | join [search index=snort | dedup src_ip,signature | table src_ip,signature]

However, if I run the search below, dedup works as expected and the combination of src_ip 1.2.3.4 and signature X only shows up once

index=snort src_ip=1.2.3.4 | dedup src_ip,signature | table src_ip signature

[vasanthmss]

Hi,

index=http status=200 src_ip!=10.0.0.0/8 src_ip!=192.168.0.0/16 src_ip!=172.16.0.0/12 | table src_ip

The above query will gives you all the possible src_ip details (including duplicate entries). Then you are left joining with other index(duplicate remove).

So either you need to remove the duplicate before or after join. so the query will be like this,

index=http status=200 src_ip!=10.0.0.0/8 src_ip!=192.168.0.0/16 src_ip!=172.16.0.0/12 | dedup src_ip | table src_ip | join [search index=snort | dedup src_ip,signature | table src_ip,signature]

Thanks,
V

[NOUMSSI]

hi try this:

index=http status=200 src_ip!=10.0.0.0/8 src_ip!=192.168.0.0/16 src_ip!=172.16.0.0/12 | dedup src_ip| join [search index=snort | dedup signature | table src_ip,signature]

[ngatchasandra]

Hi Erra,

try with

index=http status=200 src_ip!=10.0.0.0/8 src_ip!=192.168.0.0/16 src_ip!=172.16.0.0/12 | table src_ip | join [search index=snort | table src_ip,signature | dedup src_ip,signature ]

[woodcock · Accepted solution]

It is because you are joining the first dataset (which has multiples) with the second one (which is deduped). Try this:

index=http status=200 src_ip!=10.0.0.0/8 src_ip!=192.168.0.0/16 src_ip!=172.16.0.0/12 | table src_ip | join [search index=snort] | dedup src_ip,signature | table src_ip,signature

P.S. You would not be having this trouble if you had used my stats-based solution to your previous question! 😆

[ErraticIncome93]

I was working my way through both answers 😃

thanks for the help!!