Splunk Search

Filter based on groups of keys

machosplunker
Explorer

I have logs which contains keys like this.

Concept1

key=/UUID:uuid1/concept1:100
key=/UUID:uuid2/concept1:123
..
key=/UUID:uuid3/concept1:456

Concept 2

key=/UUID:uuid1/concept2:200
key=/UUID:uuid2/concept2:201
..
key=/UUID:uuid3/concept2:202

Concept 3

key=/UUID:uuid3/concept3:301
key=/UUID:uuid4/concept3:301
..
key=/UUID:uuid5/concept3:302

Is it possible to filter the results to filter only keys containing concept1.

I am thinking about using something like key="\/UUID:/concept1:" would this work?

Tags (3)
0 Karma
1 Solution

jonuwz
Influencer

This would work.

... | where match(key,"/UUID:\w+/concept1") | ...

but this would pull back all events matching the 1st part of the search ( the ... ) then filter everything else out.

It would be more efficient to do :

concept1 | where match(key,"/UUID:\w+/concept1") | ...

so this pulls out everything that contains concept1 (its a word - splunk works on words in the initial search - where a word is something surrounded by whitespace or punctuation), And then it would filter more specifically.

or maybe just this would suffice.

key=*concept1

View solution in original post

jonuwz
Influencer

This would work.

... | where match(key,"/UUID:\w+/concept1") | ...

but this would pull back all events matching the 1st part of the search ( the ... ) then filter everything else out.

It would be more efficient to do :

concept1 | where match(key,"/UUID:\w+/concept1") | ...

so this pulls out everything that contains concept1 (its a word - splunk works on words in the initial search - where a word is something surrounded by whitespace or punctuation), And then it would filter more specifically.

or maybe just this would suffice.

key=*concept1

bjoernjensen
Contributor

I think the best way to filter data before index time is discribed here

Filtering could also be done within a heavy forwarder. Just to reduce your workload asap. So mapping to your context:

Keep specific events and discard the rest

(1) In props.conf:

[source::]
TRANSFORMS-set= setnull,setparsing

(2) In transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = key=/UUID:some_string_value/concept1:some_number_value
DEST_KEY = queue
FORMAT = indexQueue

0 Karma

machosplunker
Explorer

I edited the question, might not have been clear with what i needed.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...