I have a column called "FINAL" which contains :
FINAL
kolkata,mumbai,kolkata
delhi,mumbai
delhi,delhi
hydrabad
hydrabad
My output should be :
FINAL
kolkata,mumbai
delhi,mumbai
delhi
hydrabad
hydrabad
I just want to remove the duplicate value from each row of "FINAL" filed..
Please help !!
Thanks in Advance,
Abhay
[all caps title edited]
So If the FINAL field has multiple values on each event
event1 : FINAL="kolkata,mumbai"
event2 : FINAL="delhi,mumbai"
event3 : FINAL="delhi"
event4 : FINAL="hydrabad"
event5 : FINAL="hydrabad"
Then you should make sure that the fields contains multivalues, if not, make sure to extract them.
Here is a way to remoce the comma, then remove the duplicated per event, and recombine the multivalue event.
* | makemv delim="," FINAL
| stats values(FINAL) AS FINAL by _raw
| nomv FINAL
| table FINAL
Have a look at the "dedup" search command (http://docs.splunk.com/Documentation/Splunk/5.0/SearchReference/dedup):
your search | fields FINAL | dedup FINAL
//edit: Sorry, I didn't see that you wan't remove duplicate inside the field. Is it a multi-value field or just a simple string? Do you concatenate the FINAL field first? Maybe you could use dedup before concatenating.
This is a multi-value string not a simple string...Please help me achieve this !!
Thanks in Advance
Abhay