Splunk Search
Highlighted

Filter based on groups of keys

Explorer

I have logs which contains keys like this.

Concept1

key=/UUID:uuid1/concept1:100
key=/UUID:uuid2/concept1:123
..
key=/UUID:uuid3/concept1:456

Concept 2

key=/UUID:uuid1/concept2:200
key=/UUID:uuid2/concept2:201
..
key=/UUID:uuid3/concept2:202

Concept 3

key=/UUID:uuid3/concept3:301
key=/UUID:uuid4/concept3:301
..
key=/UUID:uuid5/concept3:302

Is it possible to filter the results to filter only keys containing concept1.

I am thinking about using something like key="\/UUID:/concept1:" would this work?

Tags (3)
0 Karma
Highlighted

Re: Filter based on groups of keys

Contributor

I think the best way to filter data before index time is discribed here

Filtering could also be done within a heavy forwarder. Just to reduce your workload asap. So mapping to your context:

Keep specific events and discard the rest

(1) In props.conf:

[source::]
TRANSFORMS-set= setnull,setparsing

(2) In transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = key=/UUID:somestringvalue/concept1:somenumbervalue
DEST_KEY = queue
FORMAT = indexQueue

0 Karma
Highlighted

Re: Filter based on groups of keys

Explorer

I edited the question, might not have been clear with what i needed.

0 Karma
Highlighted

Re: Filter based on groups of keys

Influencer

This would work.

... | where match(key,"/UUID:\w+/concept1") | ...

but this would pull back all events matching the 1st part of the search ( the ... ) then filter everything else out.

It would be more efficient to do :

concept1 | where match(key,"/UUID:\w+/concept1") | ...

so this pulls out everything that contains concept1 (its a word - splunk works on words in the initial search - where a word is something surrounded by whitespace or punctuation), And then it would filter more specifically.

or maybe just this would suffice.

key=*concept1

View solution in original post