Archive

Can you help me with a query that removes duplicate events?

maheshsat
Explorer

I want to suppress duplicate events. I am trying the below command, but I'm unable to remove duplicate events.

index="id_sp" sourcetype="current" | dedup _raw | search  Period = "Jun-17"  Prepare="Actual" Fixed ="12345"  | rex "(.sap=\"(?[^\",]+))" | table Period  

Note i don't have any fields in which value get changed.

Tags (1)
0 Karma

bjoernjensen
Contributor

Hey,

your table (at the end of your spl) has just Period as column. Since you do a dedup on _raw and search for a certain (no wildcard) Prepare there almost must be only dublicate entries. Moreover your search is optimizable, and maybe you want to add more columns to the table:

index="id_sp" sourcetype="current" Period = "Jun-17" Prepare="Actual" Fixed ="12345"
| dedup _raw
| rex ".sap=(?<sap>\"(?[^\",]+))"
| table Period Prepare Fixed sap

Cheerz,
Björn

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!