Splunk Search

dedup on values of a field that match certain regular expressions

cmak
Contributor

I have a few different values for a Status field that match a certain regular expression that I would like to dedup on.

The following values are possible values for Status:
Active
Resolved *
Closed *

the * indicates wildcard, such as
Resolved (Fixed), Closed (Completed)

I would like a way to dedup on Status so that it can yield a max of 2 events.
Active and either Resolved * or Closed *.

Therefore, if I had 5 events with the following status
Active
Resolved (Fixed)
Resolved (Completed)
Closed (Fixed)
Closed (Completed)

I would like to call dedup on Status field and have only :
Active
Closed (Fixed)

I arbitrarily chose Closed (fixed) as an example to keep. I do not really care, I just would like to keep one of them (i suppose either the earliest or latest one in time would be a good standard).

0 Karma
1 Solution

cmak
Contributor

I have one solution (not sure how good it is) using replace statements

eval simple_status=sStatus|replace Resolved* with Resolved in simple_status| replace Closed* with Resolved in simple_status|dedup simple_status|

View solution in original post

0 Karma

jonuwz
Influencer

This will replace the value of simple_status with the 1st word in the original status

... | rex field=simple_status "^(?<simple_status>\S+).*" | ...

Edit

or if you want Resolved* and Closed* to resolve to the same thing :

... | eval simple_status=if(match(simple_status,"^Resolved.*|^Closed.*"),"Resolved",simple_status)) | ...
0 Karma

jonuwz
Influencer

Sorry - yeah - fixed the typo - and sorry, i misread your question. Alternative solution in answer

0 Karma

cmak
Contributor

Is there a syntax error? It gives me this error:

Error in 'rex' command: The regex '^(<?simple_status>\S+).*' does not extract anything. It should specify at least one named group. Format: (?...).

Also, I would still have two different fields (Closed and Resolved) when I want them to be identical

0 Karma

cmak
Contributor

I have one solution (not sure how good it is) using replace statements

eval simple_status=sStatus|replace Resolved* with Resolved in simple_status| replace Closed* with Resolved in simple_status|dedup simple_status|
0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...