Splunk Search

dedup on values of a field that match certain regular expressions

cmak
Contributor

I have a few different values for a Status field that match a certain regular expression that I would like to dedup on.

The following values are possible values for Status:
Active
Resolved *
Closed *

the * indicates wildcard, such as
Resolved (Fixed), Closed (Completed)

I would like a way to dedup on Status so that it can yield a max of 2 events.
Active and either Resolved * or Closed *.

Therefore, if I had 5 events with the following status
Active
Resolved (Fixed)
Resolved (Completed)
Closed (Fixed)
Closed (Completed)

I would like to call dedup on Status field and have only :
Active
Closed (Fixed)

I arbitrarily chose Closed (fixed) as an example to keep. I do not really care, I just would like to keep one of them (i suppose either the earliest or latest one in time would be a good standard).

0 Karma
1 Solution

cmak
Contributor

I have one solution (not sure how good it is) using replace statements

eval simple_status=sStatus|replace Resolved* with Resolved in simple_status| replace Closed* with Resolved in simple_status|dedup simple_status|

View solution in original post

0 Karma

jonuwz
Influencer

This will replace the value of simple_status with the 1st word in the original status

... | rex field=simple_status "^(?<simple_status>\S+).*" | ...

Edit

or if you want Resolved* and Closed* to resolve to the same thing :

... | eval simple_status=if(match(simple_status,"^Resolved.*|^Closed.*"),"Resolved",simple_status)) | ...
0 Karma

jonuwz
Influencer

Sorry - yeah - fixed the typo - and sorry, i misread your question. Alternative solution in answer

0 Karma

cmak
Contributor

Is there a syntax error? It gives me this error:

Error in 'rex' command: The regex '^(<?simple_status>\S+).*' does not extract anything. It should specify at least one named group. Format: (?...).

Also, I would still have two different fields (Closed and Resolved) when I want them to be identical

0 Karma

cmak
Contributor

I have one solution (not sure how good it is) using replace statements

eval simple_status=sStatus|replace Resolved* with Resolved in simple_status| replace Closed* with Resolved in simple_status|dedup simple_status|
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...