Splunk Search

How to remove duplicates?

wanda619
Path Finder

Client

Error

Error Results

Error ResultsPrevious week

Percent of Total

PercentDifference

abc

1003

2

0

12.5

0

abc

1003

3

 

12.5

0

abc

1013

1

2

342

-50

abc

1027

3

3

5

0

abc

1027

5

xyz

43

zyz

abc

1013

2

zyz

432

et

abc

Total

16

zyds

423

tert

 

My code is   -- 

 

| bucket _time span=1w
| stats count as Result by LicenseKey, Error_Code
| eval Client=coalesce(Client,LicenseKey)
| eventstats sum(Result) as Total by Client
| eval PercentOfTotal = round((Result/Total)*100,3)
| sort - _time
| streamstats current=f latest(Result) as Result_Prev by LicenseKey
| eval PercentDifference = round(((Result/Result_Prev)-1)*100,2)
| fillnull value="0"
| append
    [ search index=abc sourcetype=yxx source= bff ErrorCode!=0
    | `DedupDHI`
    | lookup abc LicenseKey OUTPUT Client
    | eval Client=coalesce(Client,LicenseKey)
    | stats count as Result by Client
    | eval ErrorCode="Total", PercentOfTotal=100]
| lookup xyz_ErrorCodes ErrorCode OUTPUT Description
| lookup uyz LicenseKey OUTPUT Client
| eval Client=coalesce(Client,LicenseKey)
| eval Error=if(ErrorCode!="Total", ErrorCode+" ("+coalesce(Description,"Description Missing - Update xyz_ErrorCodes")+")", ErrorCode)
| fields Client, Error, Result, PercentOfTotal, PercentDifference, Error results previous week
| sort CustomerName, Error, PercentDifference

 

Still not able to figure out the duplicate row issue, single row for one each error combined with total.

any suggestions please? 

Labels (7)
Tags (2)
0 Karma

bowesmana
SplunkTrust
SplunkTrust

I believe you already asked this 

https://community.splunk.com/t5/Splunk-Search/How-to-remove-duplicates-from-the-table/m-p/608750

Do you want to remove all other fields in the table other than Error? What should occur with these two rows

abc

1027

3

3

5

0

abc

1027

5

xyz

43

zyz

 

for previous week and other columns?

Simple answer would be

| stats sum("Error Results") as "Error Results" values(*) as * by Client Error

but you need to a bit clearer about the removal part of duplicates.

The above would sum the error counts and just show all the values collected in the same row as the error.

I am assuming your sorting by CustomerName in your search is not what you meant/wanted, as there is no CustomerName field in your data

 

richgalloway
SplunkTrust
SplunkTrust

Please explain or provide a mock-up of the expected results.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...