I have logs which contains keys like this.
Concept1
key=/UUID:uuid1/concept1:100
key=/UUID:uuid2/concept1:123
..
key=/UUID:uuid3/concept1:456
Concept 2
key=/UUID:uuid1/concept2:200
key=/UUID:uuid2/concept2:201
..
key=/UUID:uuid3/concept2:202
Concept 3
key=/UUID:uuid3/concept3:301
key=/UUID:uuid4/concept3:301
..
key=/UUID:uuid5/concept3:302
Is it possible to filter the results to filter only keys containing concept1.
I am thinking about using something like key="\/UUID:/concept1:" would this work?
This would work.
... | where match(key,"/UUID:\w+/concept1") | ...
but this would pull back all events matching the 1st part of the search ( the ... ) then filter everything else out.
It would be more efficient to do :
concept1 | where match(key,"/UUID:\w+/concept1") | ...
so this pulls out everything that contains concept1 (its a word - splunk works on words in the initial search - where a word is something surrounded by whitespace or punctuation), And then it would filter more specifically.
or maybe just this would suffice.
key=*concept1
This would work.
... | where match(key,"/UUID:\w+/concept1") | ...
but this would pull back all events matching the 1st part of the search ( the ... ) then filter everything else out.
It would be more efficient to do :
concept1 | where match(key,"/UUID:\w+/concept1") | ...
so this pulls out everything that contains concept1 (its a word - splunk works on words in the initial search - where a word is something surrounded by whitespace or punctuation), And then it would filter more specifically.
or maybe just this would suffice.
key=*concept1
I think the best way to filter data before index time is discribed here
Filtering could also be done within a heavy forwarder. Just to reduce your workload asap. So mapping to your context:
Keep specific events and discard the rest
(1) In props.conf:
[source::]
TRANSFORMS-set= setnull,setparsing
(2) In transforms.conf:
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[setparsing]
REGEX = key=/UUID:some_string_value/concept1:some_number_value
DEST_KEY = queue
FORMAT = indexQueue
I edited the question, might not have been clear with what i needed.