Filter based on groups of keys

[machosplunker]

I have logs which contains keys like this.

Concept1

key=/UUID:uuid1/concept1:100
key=/UUID:uuid2/concept1:123
..
key=/UUID:uuid3/concept1:456

Concept 2

key=/UUID:uuid1/concept2:200
key=/UUID:uuid2/concept2:201
..
key=/UUID:uuid3/concept2:202

Concept 3

key=/UUID:uuid3/concept3:301
key=/UUID:uuid4/concept3:301
..
key=/UUID:uuid5/concept3:302

Is it possible to filter the results to filter only keys containing concept1.

I am thinking about using something like key="\/UUID:/concept1:" would this work?

[jonuwz · Accepted solution]

This would work.

... | where match(key,"/UUID:\w+/concept1") | ...

but this would pull back all events matching the 1st part of the search ( the ... ) then filter everything else out.

It would be more efficient to do :

concept1 | where match(key,"/UUID:\w+/concept1") | ...

so this pulls out everything that contains concept1 (its a word - splunk works on words in the initial search - where a word is something surrounded by whitespace or punctuation), And then it would filter more specifically.

or maybe just this would suffice.

key=*concept1

[bjoernjensen]

I think the best way to filter data before index time is discribed here

Filtering could also be done within a heavy forwarder. Just to reduce your workload asap. So mapping to your context:

Keep specific events and discard the rest

(1) In props.conf:

[source::]
TRANSFORMS-set= setnull,setparsing

(2) In transforms.conf:

[setnull]

REGEX = .

DEST_KEY = queue

FORMAT = nullQueue

[setparsing]
REGEX = key=/UUID:some_string_value/concept1:some_number_value
DEST_KEY = queue
FORMAT = indexQueue

[machosplunker]

I edited the question, might not have been clear with what i needed.