topic Re: Negation in Transforms.conf in Getting Data In

Negation in Transforms.conf

abhayneilam — Mon, 28 Sep 2020 18:38:19 GMT

Hi,
I have a lot in which the below lines are coming

AAA good
BBB good
AAA good
CCC good
DDD bad
AAA bad

I want to nullify the lines which does not have "AAA good" sting. please help me in writing the transforms.conf :

[AAA_GOOD]
REGEX = (?!)AAA\sgood
DEST_KEY = queue
FORMAT = nullQueue

Please help !!

Re: Negation in Transforms.conf

MuS — Thu, 15 Jan 2015 12:11:54 GMT

Hi abhayneilam,

use this regex and it will only match AAA good:

^(?!.*AAA\sgood$).*

cheers, MuS

Re: Negation in Transforms.conf

abhayneilam — Thu, 15 Jan 2015 12:15:43 GMT

If my value is something like :

AAA\s[good\sverygood\swow]

then it should be like

[^AAA\s[good.verygood.wow]] right ? please suggest !!

Re: Negation in Transforms.conf

MuS — Thu, 15 Jan 2015 12:26:38 GMT

those \s literal or regex spaces?

Re: Negation in Transforms.conf

abhayneilam — Thu, 15 Jan 2015 12:29:25 GMT

I am giving \s for denoting spaces and also giving \ before .

I have a string like

blah blah AAA [good.verygood.wow] blah blah

Re: Negation in Transforms.conf

MuS — Thu, 15 Jan 2015 13:29:16 GMT

How about the docs approach of nulling everything and explicitly keep only the AAA [good events like this:

In props.conf:

[AAA_GOOD]
TRANSFORMS-set= setnull,setparsing

In transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = AAA\s\[good
DEST_KEY = queue
FORMAT = indexQueue

I tend to setup things the way you can remember what they mean in two years from now.

Re: Negation in Transforms.conf

abhayneilam — Mon, 28 Sep 2020 18:38:22 GMT

I have done :

in props.conf

[AAA_GOOD]
SHOULD_LINEMERGE = false
TRANSFORMS-set = setnull,setparsing,add_host,BBB_MORE

in transforms.conf

[setnull]
REGEX= .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = AAA\s[good
DEST_KEY = queue
FORMAT = AAA_Index

[add_host]
SOURCE_KEY = MetaData:Host
REGEX = ^host::(.*)$
FORMAT = HS="$1" $0
DEST_KEY = _raw

[BBB_MORE]
REGEX=^(.?)\s(\d+-\d+-\d+)\s(\d+:\d+:\d+,\d+)\sAAA\s[good]\s[\d+]\s(\w+)\s::\s(.)
FORMAT=$1 AA=aa BB=bb CC="$2T$3" DD=$4 EE=$5
DEST_KEY = _raw

Please let me know if this is correct.

I am using my own index name "AAA_Index" instead of "indexQueue" in props.conf suggested by you. Is it correct ?

Re: Negation in Transforms.conf

MuS — Thu, 15 Jan 2015 14:12:50 GMT

No that's wrong, you cannot specify an index in this case, because the DEST_KEY is queue. If you want to rewrite the index you must use this:

DEST_KEY = _MetaData:Index
FORMAT = AAA_Index

And as hint: provide such information in the initial question 😉 makes it easier to answer 🙂