topic Re: How to omit duplicate values of a column in Getting Data In

How to omit duplicate values of a column

harish_ka — Fri, 12 Dec 2014 14:43:05 GMT

Hi i have a report as below,

Col A -----Col B--------Col C-----Col D
-------------------------------------------------
ABCD-----US  ----------323------12
XYZZ------AM-----------323------11
SADF-----SD------------323-------88

i need to remove the duplicate values of col C, i need to show only once in the first row like below,

Col A -----Col B--------Col C-----Col D
-------------------------------------------------
ABCD-----US  ----------323------12
XYZZ------AM----------------------11
SADF-----SD-----------------------88

Here the report is grouped by ColA and ColB...

Can anyone help me on this....

Re: How to omit duplicate values of a column

somesoni2 — Fri, 12 Dec 2014 15:45:08 GMT

What should be the output if report is like this.

 Col A -----Col B--------Col C-----Col D
-------------------------------------------------
ABCD-----US ----------323------12
XYZZ------AM-----------323------11
SADF-----SD------------323-------88
PQRS------PM-----------999------11
QWER-----GB------------323-------88

Re: How to omit duplicate values of a column

harish_ka — Fri, 12 Dec 2014 15:55:08 GMT

The column C values remains the same...
if i have 5 rows..all values of Col C will be 323

Re: How to omit duplicate values of a column

somesoni2 — Fri, 12 Dec 2014 16:10:08 GMT

Based on your current update, this should work.

your base search giving "Col C" and "Col D" group By "Col A" and "Col B"  | streamstats count as sno by "Col C"| eventstats max(sno) as max count as total  | eval "Col C"=if(total>max,'Col C', if(sno=1,'Col C', null())) | fields - sno,max,total

Re: How to omit duplicate values of a column

rsathish47 — Mon, 28 Sep 2020 18:26:40 GMT

base search | streamstats count as sno by Col_C | eval Col_C=if(sno=1,Col_C,"") | fields - sno

Re: How to omit duplicate values of a column

chimell — Mon, 15 Dec 2014 08:48:56 GMT

Try this search code I ‘m sure that it will be work well

replace omitduplication.csv by a source name of your file

source="omitduplication.csv "| stats count by colA ,colB, colD | appendcols[search source="omitduplication.csv" | dedup colC] | table colA colB colC colD

Re: How to omit duplicate values of a column

harish_ka — Mon, 15 Dec 2014 09:52:44 GMT

Thank you so much somesoni2 and rsatish47 🙂
its working as i expected...