Splunk Search

How to remove duplicates?

wanda619
Path Finder

Client

Error

Error Results

Error ResultsPrevious week

Percent of Total

PercentDifference

abc

1003

2

0

12.5

0

abc

1003

3

 

12.5

0

abc

1013

1

2

342

-50

abc

1027

3

3

5

0

abc

1027

5

xyz

43

zyz

abc

1013

2

zyz

432

et

abc

Total

16

zyds

423

tert

 

My code is   -- 

 

| bucket _time span=1w
| stats count as Result by LicenseKey, Error_Code
| eval Client=coalesce(Client,LicenseKey)
| eventstats sum(Result) as Total by Client
| eval PercentOfTotal = round((Result/Total)*100,3)
| sort - _time
| streamstats current=f latest(Result) as Result_Prev by LicenseKey
| eval PercentDifference = round(((Result/Result_Prev)-1)*100,2)
| fillnull value="0"
| append
    [ search index=abc sourcetype=yxx source= bff ErrorCode!=0
    | `DedupDHI`
    | lookup abc LicenseKey OUTPUT Client
    | eval Client=coalesce(Client,LicenseKey)
    | stats count as Result by Client
    | eval ErrorCode="Total", PercentOfTotal=100]
| lookup xyz_ErrorCodes ErrorCode OUTPUT Description
| lookup uyz LicenseKey OUTPUT Client
| eval Client=coalesce(Client,LicenseKey)
| eval Error=if(ErrorCode!="Total", ErrorCode+" ("+coalesce(Description,"Description Missing - Update xyz_ErrorCodes")+")", ErrorCode)
| fields Client, Error, Result, PercentOfTotal, PercentDifference, Error results previous week
| sort CustomerName, Error, PercentDifference

 

Still not able to figure out the duplicate row issue, single row for one each error combined with total.

any suggestions please? 

Labels (7)
Tags (2)
0 Karma

bowesmana
SplunkTrust
SplunkTrust

I believe you already asked this 

https://community.splunk.com/t5/Splunk-Search/How-to-remove-duplicates-from-the-table/m-p/608750

Do you want to remove all other fields in the table other than Error? What should occur with these two rows

abc

1027

3

3

5

0

abc

1027

5

xyz

43

zyz

 

for previous week and other columns?

Simple answer would be

| stats sum("Error Results") as "Error Results" values(*) as * by Client Error

but you need to a bit clearer about the removal part of duplicates.

The above would sum the error counts and just show all the values collected in the same row as the error.

I am assuming your sorting by CustomerName in your search is not what you meant/wanted, as there is no CustomerName field in your data

 

richgalloway
SplunkTrust
SplunkTrust

Please explain or provide a mock-up of the expected results.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...