Splunk Search

Compare rows within a table for duplicates

rlautman
Path Finder

Hi,

I have created a report that takes a lookup list of order references and returns all other orders that are related, my problem is that a lot of the results from my search are duplicates - I cannot use a dedup command on any of the fields as I may miss some results - so what I wanted to do was to compare 2 fields within the rows - e.g. OrderID and CompletionStatus - check that there is no row with duplicated information, and remove 1 of the rows if there is. I have seen a similar issue here but this looks like it depends on there being only 2 rows. Is what I am asking possible and if so can anyone suggest how I would go about doing this?

0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

I guess I don't see why you can't use

... | dedup OrderID CompletionStatus

which will keep only one of each combination?

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

I guess I don't see why you can't use

... | dedup OrderID CompletionStatus

which will keep only one of each combination?

rlautman
Path Finder

This seems to do the job, thanks, I wasn't sure if dedup could be used for multiple fields 🙂

0 Karma

kristian_kolb
Ultra Champion

Could the combination of multikv and dedup not be of help?

From the docs on multikv:
Extracts fields from events with information in a tabular format (e.g. top, netstat, ps, ... etc). A new event will be created for each table row. Field names will be derived from the title row of the table.

Then use | dedup field1 field2 field3 to only keep unique combinations of the three fields' values.

/K

0 Karma
Get Updates on the Splunk Community!

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

🔐 Trust at Every Hop: How mTLS in Splunk Enterprise 10.0 Makes Security Simpler

From Idea to Implementation: Why Splunk Built mTLS into Splunk Enterprise 10.0  mTLS wasn’t just a checkbox ...