Splunk Search

How to remove duplicates?

wanda619
Path Finder

Client

Error

Error Results

Error ResultsPrevious week

Percent of Total

PercentDifference

abc

1003

2

0

12.5

0

abc

1003

3

 

12.5

0

abc

1013

1

2

342

-50

abc

1027

3

3

5

0

abc

1027

5

xyz

43

zyz

abc

1013

2

zyz

432

et

abc

Total

16

zyds

423

tert

 

My code is   -- 

 

| bucket _time span=1w
| stats count as Result by LicenseKey, Error_Code
| eval Client=coalesce(Client,LicenseKey)
| eventstats sum(Result) as Total by Client
| eval PercentOfTotal = round((Result/Total)*100,3)
| sort - _time
| streamstats current=f latest(Result) as Result_Prev by LicenseKey
| eval PercentDifference = round(((Result/Result_Prev)-1)*100,2)
| fillnull value="0"
| append
    [ search index=abc sourcetype=yxx source= bff ErrorCode!=0
    | `DedupDHI`
    | lookup abc LicenseKey OUTPUT Client
    | eval Client=coalesce(Client,LicenseKey)
    | stats count as Result by Client
    | eval ErrorCode="Total", PercentOfTotal=100]
| lookup xyz_ErrorCodes ErrorCode OUTPUT Description
| lookup uyz LicenseKey OUTPUT Client
| eval Client=coalesce(Client,LicenseKey)
| eval Error=if(ErrorCode!="Total", ErrorCode+" ("+coalesce(Description,"Description Missing - Update xyz_ErrorCodes")+")", ErrorCode)
| fields Client, Error, Result, PercentOfTotal, PercentDifference, Error results previous week
| sort CustomerName, Error, PercentDifference

 

Still not able to figure out the duplicate row issue, single row for one each error combined with total.

any suggestions please? 

Labels (7)
Tags (2)
0 Karma

bowesmana
SplunkTrust
SplunkTrust

I believe you already asked this 

https://community.splunk.com/t5/Splunk-Search/How-to-remove-duplicates-from-the-table/m-p/608750

Do you want to remove all other fields in the table other than Error? What should occur with these two rows

abc

1027

3

3

5

0

abc

1027

5

xyz

43

zyz

 

for previous week and other columns?

Simple answer would be

| stats sum("Error Results") as "Error Results" values(*) as * by Client Error

but you need to a bit clearer about the removal part of duplicates.

The above would sum the error counts and just show all the values collected in the same row as the error.

I am assuming your sorting by CustomerName in your search is not what you meant/wanted, as there is no CustomerName field in your data

 

richgalloway
SplunkTrust
SplunkTrust

Please explain or provide a mock-up of the expected results.

---
If this reply helps you, Karma would be appreciated.
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Observability Simplified: Combining User Experience, Application Performance & ...

Tech Talk Observability Simplified: Combining User Experience, Application Performance & Network ...

Event Series May & June: From Network Visibility to Service Intelligence

Unifying the Network: Moving from Alert Noise to Service Intelligence with Splunk ITSI In today’s hybrid ...

Global Splunk User Group Events: May + June 2026

Your Splunk Community Awaits: Discover Upcoming User Group Events Worldwide    Staying ahead in the fast-paced ...