I have a question on how to restrict what goes into an index.
I have read a number of posts and documentation on how this should work.
In my case I have tried a number of permutations of the props.conf and transforms.conf with no success.
I oneshot the logs in and all the items in the log goes into the index. Here are my props.conf and transforms.conf. Any help would be great.
Thanks
V 1
[1033NCL11O]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n])
KV_MODE=none
SEGMENTATION-all = inner
TRANSFORMS-set = setnullO
TRANSFORMS-set = setparsingO
TRANSFORMS-servicename = extract-webdata-sernmO
[setnullO]
REGEX = (INFO|DEBUG)
SOURCE_KEY = queue
FORMAT = nullQueue
[setparsingO]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
V 2
[1033NCL11O]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n])
KV_MODE=none
SEGMENTATION-all = inner
TRANSFORMS-set = setparsingO
TRANSFORMS-set = setnullO
TRANSFORMS-servicename = extract-webdata-sernmO
[setnullO]
REGEX = .
SOURCE_KEY = queue
FORMAT = nullQueue
[setparsingO]
REGEX = (TEST|ERROR|WARN|ABT|DEBUG2)
DEST_KEY = queue
FORMAT = indexQueue
So what you are saying is that the issue is with the spaces around the equals.
DEST_KEY = queue should be DEST_KEY=queue
Does the spaces around the equals make a difference? If so why?
So what you are saying is that the issue is with the spaces around the equals.
DEST_KEY = queue should be DEST_KEY=queue
Does the spaces around the equals make a difference? If so why?
Why not simply
props.conf
[1033NCL11O]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n])
KV_MODE=none
SEGMENTATION-all = inner
TRANSFORMS-set = setnullO
TRANSFORMS-servicename = extract-webdata-sernmO
transforms.conf
[setnullO]
REGEX = (INFO|DEBUG)
DEST_KEY = queue
FORMAT = nullQueue
And the real problem is that it should be DEST_KEY=queue
No the real problem is that you used SOURCE_KEY = queue
instead of DEST_KEY = queue
in the setnull0
transform...
So what you are saying is that the issue is with the spaces around the equals.
DEST_KEY = queue should be DEST_KEY=queue
Does spaces around the equals make a difference? If so why?