https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217968#M64059 <P>We have devices that generate thousands of a particular entry. I created a daily search to summarize. I combined the src_int and dest_int into a single field labeled interfaces. What my boss wants is to see the total number of events per host, but only unique to the new field. The problem is he also wants to dedup the interfaces field even if the src_int and dest_int are reversed like this:<BR /> <IMG src="https://community.splunk.com/storage/temp/175172-dedup.jpg" alt="alt text" /></P> <P>Here is the current search:</P> <PRE><CODE>index=network sourcetype="cisco:ios*" (key_word="MAC_MOVE-SP-4-NOTIF" OR key_word="MAC_MOVE-SW1-4-NOTIF") | eval Interfaces = src_int + "," + dest_int | table host, Interfaces | sort host | dedup Interfaces </CODE></PRE> <P>What I need is to somehow dedup the "Interfaces" field even if the 2 fields that make up the eval are reversed. I then need to add a count of unique "Interfaces" entries per host after the dedup. </P> <P>Thanks!</P> Tue, 29 Sep 2020 12:17:20 GMT mvasquez2 2020-09-29T12:17:20Z https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217968#M64059 <P>We have devices that generate thousands of a particular entry. I created a daily search to summarize. I combined the src_int and dest_int into a single field labeled interfaces. What my boss wants is to see the total number of events per host, but only unique to the new field. The problem is he also wants to dedup the interfaces field even if the src_int and dest_int are reversed like this:<BR /> <IMG src="https://community.splunk.com/storage/temp/175172-dedup.jpg" alt="alt text" /></P> <P>Here is the current search:</P> <PRE><CODE>index=network sourcetype="cisco:ios*" (key_word="MAC_MOVE-SP-4-NOTIF" OR key_word="MAC_MOVE-SW1-4-NOTIF") | eval Interfaces = src_int + "," + dest_int | table host, Interfaces | sort host | dedup Interfaces </CODE></PRE> <P>What I need is to somehow dedup the "Interfaces" field even if the 2 fields that make up the eval are reversed. I then need to add a count of unique "Interfaces" entries per host after the dedup. </P> <P>Thanks!</P> Tue, 29 Sep 2020 12:17:20 GMT https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217968#M64059 mvasquez2 2020-09-29T12:17:20Z https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217969#M64060 <P>Try something like this</P> <PRE><CODE>index=network sourcetype="cisco:ios*" (key_word="MAC_MOVE-SP-4-NOTIF" OR key_word="MAC_MOVE-SW1-4-NOTIF") | eval Interfaces = mvsort(split(src_int + "," + dest_int,","))| nomv interfaces | table host, Interfaces | sort host | dedup Interfaces </CODE></PRE> Tue, 03 Jan 2017 19:19:34 GMT https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217969#M64060 somesoni2 2017-01-03T19:19:34Z https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217970#M64061 <P>OK. Almost there. This is awesome. All I need now is to add a column with the count of the number of events before the dedup.</P> <P>thanks!</P> Tue, 03 Jan 2017 20:42:21 GMT https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217970#M64061 mvasquez2 2017-01-03T20:42:21Z https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217971#M64062 <P>actually it would be the # of events AFTER the dedup</P> Tue, 03 Jan 2017 21:11:43 GMT https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217971#M64062 mvasquez2 2017-01-03T21:11:43Z https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217972#M64063 <P>Try this</P> <PRE><CODE> index=network sourcetype="cisco:ios*" (key_word="MAC_MOVE-SP-4-NOTIF" OR key_word="MAC_MOVE-SW1-4-NOTIF") | eval Interfaces = mvsort(split(src_int + "," + dest_int,","))| nomv interfaces | table host, Interfaces | sort host | dedup Interfaces | eventstats count </CODE></PRE> Tue, 03 Jan 2017 22:59:42 GMT https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217972#M64063 somesoni2 2017-01-03T22:59:42Z https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217973#M64064 <P>I had tried that but it gives the same total for each line:</P> <P>Host Interfaces Count<BR /> ns-s-972brus-6509c Gi7/37 47<BR /> Po246<BR /><BR /> ns-s-972brus-6509c Gi7/48 47<BR /> Po246<BR /><BR /> ns-s-972brus-6509c Gi4/25 47<BR /> Po246<BR /><BR /> ns-s-972brus-6509c Gi4/23 47<BR /> Po246 </P> <P>What I need is the count of the number of events for each pair of interfaces. For this search the total number of events was 47 so I would expect the count for each pair to add up to 47 total.</P> <P>Thanks!</P> Wed, 04 Jan 2017 14:58:07 GMT https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217973#M64064 mvasquez21 2017-01-04T14:58:07Z https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217974#M64065 <P>I should've pointed this one out, after dedup, there will be only 1 entry available for each interface, so I guess you need the count of events what was before the dedup. Try this:</P> <PRE><CODE>index=network sourcetype="cisco:ios*" (key_word="MAC_MOVE-SP-4-NOTIF" OR key_word="MAC_MOVE-SW1-4-NOTIF") | eval Interfaces = mvsort(split(src_int + "," + dest_int,","))| nomv interfaces | stats count latest(host) as host by Interfaces | sort host </CODE></PRE> Wed, 04 Jan 2017 15:34:37 GMT https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217974#M64065 somesoni2 2017-01-04T15:34:37Z https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217975#M64066 <P>Ah. I gotcha. This works though for the count. Thanks so much for your help!</P> Wed, 04 Jan 2017 18:37:46 GMT https://community.splunk.com/t5/Splunk-Search/How-to-dedup-a-combination-of-two-fields-and-get-the-count-of/m-p/217975#M64066 mvasquez21 2017-01-04T18:37:46Z