How to removine duplicates in a single field?

[smanojkumar]

Hi There!

    Good day,

    I need to remove repeated entries of same values in single field, I'm unable to separate into single values by using values() , mvsplit commands,

Actual one - 

src_name serial

item-s1028501

5cd022g2wn 5cd022g2wn 5cd022g2wn 5cd022g2wn 5cd022g2wn 5cd022g2wn 5cd022g2wn 5cd022g2wn

Expected one -

src_name serial

item-s1028501

5cd022g2wn

I just mentioned only one values in src_name, we are having multiple values in src_name to work with,

Thanks in Advance !

[TrangCIC81]

Is the name of the field containing duplicates that you want to remove "src_name"?
if so you can use the below

<your_search> | dedup src_name

[ITWhisperer]

| eval unique=mvdedup(split(serial," "))

[smanojkumar]

Hi @ITWhisperer ,

   After applying the comamand, I got this,

The data is from lookup,

serial src_name length unique

5cd022g2wn 5cd022g2wn 5cd022g2wn 5cd022g2wn 5cd022g2wn 5cd022g2wn 5cd022g2wn

item-s1028501

2718

5cd022g2wn 5cd022g2wn 5cd022g2wn 5cd022g2wn 5cd022g2wn 5cd022g2wn 5cd022g2wn

[ITWhisperer]

The example you posted has the values separated by spaces (which is what my solution deals with). If your serial field is not actually like that, which would cause the solution to not give the expected answer, you will have to explain what you actual field contains. For example, is it already a multivalue field, or are the values separated by tabs or commas. If you don't provide an accurate representation of your data, you cannot expect an accurate solution to be provided.

[smanojkumar]

Here is the actual data in JSON format,

{"preview":false,"lastrow":true,"result":{"serial":"5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn\n5cd022g2wn  ","src_name":"item-s1028501","length":"2718"}}

[ITWhisperer]

Since you appear to have a trailing space as well as have \n delimiters, try this

| eval unique=mvdedup(split(trim(serial),"\n"))

[smanojkumar]

Hi @ITWhisperer ,

   I got the same output as previous,

Thanks in Advance!

[ITWhisperer · Accepted solution]

Not sure why the split wasn't recognising the \n - try replacing them with commas, and splitting by those

| eval unique=mvdedup(split(replace(trim(serial_number),"\n",","),","))

[smanojkumar]

Thanks @ITWhisperer , It worked.