Splunk Search

Dedup is not working with mstats

sabari80
Explorer

We are streaming Dynatrace metric data into Splunk, for some reason we are seeing duplicate 'MessageDeduplicationId'. So trying to avoid the duplicate entries using dedup command. But not retrieving any results after using dedup command. Here is my initial query and getting results for this with duplicates-

| mstats sum(calc:service.thaa_stress_requests_count_lr_tags) As "Count" ,avg(calc:service.thaa_stress_requests_lr_tags) As "Response" where index=itsi_im_metrics AND source.name="DT_NonProd_SaaS" by Dimension.id | eval Response=round((Response/1000000),2), Count=round(Count,0) | search Dimension.id IN ("*Process.aspx") After adding dedup to avoid duplicate 'MessageDeduplicationId' , no results | mstats sum(calc:service.thaa_stress_requests_count_lr_tags) As "Count" ,avg(calc:service.thaa_stress_requests_lr_tags) As "Response" where index=itsi_im_metrics AND source.name="DT_NonProd_SaaS" by Dimension.id 
| eval Response=round((Response/1000000),2), Count=round(Count,0) 
| search Dimension.id IN ("*Process.aspx") 
| dedup MessageDeduplicationId sample payload: Dimension.id: xxxProcess.aspx Dimension.name: Literal Not Found MessageDeduplicationId: a901b712889217fc194cd0446a70325e aggregation: avg entity.service.id: xxx entity.service.name:xxxx metric_name:calc:service.thaa_stress_requests_lr_tags: 1613759 resolution: 1m source.name: xxxx unit: MicroSecond
Labels (2)
Tags (1)
0 Karma

sabari80
Explorer

sample Payload

=========

 

sample payload:

Dimension.id: xxxProcess.aspx

Dimension.name: Literal Not Found

MessageDeduplicationId: a901b712889217fc194cd0446a70325e

aggregation: avg

entity.service.id: xxx

entity.service.name:xxxx

metric_name:calc:xxxx_

lr_tags: 1613759

resolution: 1m s

ource.name: xxxx

unit: MicroSecond
0 Karma

sabari80
Explorer

Modified Query

==========

 

| mstats sum(calc:service.thaa_stress_requests_count_lr_tags) As "Count" ,avg(calc:service.thaa_stress_requests_lr_tags) As "Response" where index=itsi_im_metrics AND source.name="DT_NonProd_SaaS" by Dimension.id | eval Response=round((Response/1000000),2), Count=round(Count,0) | search Dimension.id IN ("*Process.aspx") | dedup MessageDeduplicationId 
0 Karma

sabari80
Explorer

Initial Query

==========

 

| mstats sum(calc:service.thaa_stress_requests_count_lr_tags) As "Count" ,avg(calc:service.thaa_stress_requests_lr_tags) As "Response" where index=itsi_im_metrics AND source.name="DT_NonProd_SaaS" by Dimension.id | eval Response=round((Response/1000000),2), Count=round(Count,0) | search Dimension.id IN ("*Process.aspx")
0 Karma
Get Updates on the Splunk Community!

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...