Deployment Architecture

Can you help me with a query that removes duplicate events?

maheshsat
Explorer

I want to suppress duplicate events. I am trying the below command, but I'm unable to remove duplicate events.

index="id_sp" sourcetype="current" | dedup _raw | search  Period = "Jun-17"  Prepare="Actual" Fixed ="12345"  | rex "(.sap=\"(?[^\",]+))" | table Period  

Note i don't have any fields in which value get changed.

Tags (1)
0 Karma

bjoernjensen
Contributor

Hey,

your table (at the end of your spl) has just Period as column. Since you do a dedup on _raw and search for a certain (no wildcard) Prepare there almost must be only dublicate entries. Moreover your search is optimizable, and maybe you want to add more columns to the table:

index="id_sp" sourcetype="current" Period = "Jun-17" Prepare="Actual" Fixed ="12345"
| dedup _raw
| rex ".sap=(?<sap>\"(?[^\",]+))"
| table Period Prepare Fixed sap

Cheerz,
Björn

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...