topic Re: Restriking indexed items in Getting Data In

Restriking indexed items

david_fresne — Thu, 22 May 2014 14:52:23 GMT

I have a question on how to restrict what goes into an index.
I have read a number of posts and documentation on how this should work.
In my case I have tried a number of permutations of the props.conf and transforms.conf with no success.
I oneshot the logs in and all the items in the log goes into the index. Here are my props.conf and transforms.conf. Any help would be great.

Thanks

V 1

props.conf

[1033NCL11O]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n])
KV_MODE=none
SEGMENTATION-all = inner
TRANSFORMS-set = setnullO
TRANSFORMS-set = setparsingO
TRANSFORMS-servicename = extract-webdata-sernmO

transforms.conf

[setnullO]
REGEX = (INFO|DEBUG)
SOURCE_KEY = queue
FORMAT = nullQueue

[setparsingO]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

V 2

props.conf

[1033NCL11O]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n])
KV_MODE=none
SEGMENTATION-all = inner
TRANSFORMS-set = setparsingO
TRANSFORMS-set = setnullO
TRANSFORMS-servicename = extract-webdata-sernmO

transforms.conf

[setnullO]
REGEX = .
SOURCE_KEY = queue
FORMAT = nullQueue

[setparsingO]
REGEX = (TEST|ERROR|WARN|ABT|DEBUG2)
DEST_KEY = queue
FORMAT = indexQueue

Re: Restriking indexed items

lguinn2 — Thu, 22 May 2014 19:58:52 GMT

Why not simply

props.conf

[1033NCL11O]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n])
KV_MODE=none
SEGMENTATION-all = inner
TRANSFORMS-set = setnullO
TRANSFORMS-servicename = extract-webdata-sernmO

transforms.conf

[setnullO]
REGEX = (INFO|DEBUG)
DEST_KEY = queue
FORMAT = nullQueue

And the real problem is that it should be DEST_KEY=queue

Re: Restriking indexed items

david_fresne — Mon, 28 Sep 2020 16:43:38 GMT

So what you are saying is that the issue is with the spaces around the equals.

DEST_KEY = queue should be DEST_KEY=queue

Does the spaces around the equals make a difference? If so why?

Re: Restriking indexed items

david_fresne — Mon, 28 Sep 2020 16:43:41 GMT

So what you are saying is that the issue is with the spaces around the equals.

DEST_KEY = queue should be DEST_KEY=queue

Does the spaces around the equals make a difference? If so why?

Re: Restriking indexed items

david_fresne — Mon, 28 Sep 2020 16:43:44 GMT

So what you are saying is that the issue is with the spaces around the equals.

DEST_KEY = queue should be DEST_KEY=queue

Does spaces around the equals make a difference? If so why?

Re: Restriking indexed items

lguinn2 — Fri, 30 May 2014 15:45:32 GMT

No the real problem is that you used SOURCE_KEY = queue instead of DEST_KEY = queue in the setnull0 transform...