Splunk Search

Compare rows within a table for duplicates

Path Finder

Hi,

I have created a report that takes a lookup list of order references and returns all other orders that are related, my problem is that a lot of the results from my search are duplicates - I cannot use a dedup command on any of the fields as I may miss some results - so what I wanted to do was to compare 2 fields within the rows - e.g. OrderID and CompletionStatus - check that there is no row with duplicated information, and remove 1 of the rows if there is. I have seen a similar issue here but this looks like it depends on there being only 2 rows. Is what I am asking possible and if so can anyone suggest how I would go about doing this?

0 Karma
1 Solution

Splunk Employee
Splunk Employee

I guess I don't see why you can't use

... | dedup OrderID CompletionStatus

which will keep only one of each combination?

View solution in original post

Splunk Employee
Splunk Employee

I guess I don't see why you can't use

... | dedup OrderID CompletionStatus

which will keep only one of each combination?

View solution in original post

Path Finder

This seems to do the job, thanks, I wasn't sure if dedup could be used for multiple fields 🙂

0 Karma

Ultra Champion

Could the combination of multikv and dedup not be of help?

From the docs on multikv:
Extracts fields from events with information in a tabular format (e.g. top, netstat, ps, ... etc). A new event will be created for each table row. Field names will be derived from the title row of the table.

Then use | dedup field1 field2 field3 to only keep unique combinations of the three fields' values.

/K

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!