Enterprise 7.2 - Extract doesn't work, rex does

[mweissha]

Hello all,

I have a problem with one field extract that works if I use the exact regex syntax in the rex command but it fails to extract the field as an inline regex. I have not seen anything in the logs that would suggest a problem. The regex has been tested on regex101 to work properly and it was working before we upgraded to 7.2. The regex is as follows:

props.conf
EXTRACT-Action = (?:[^ \n]* ){6}(?P[\w]+)\b (?<=\B)

| rex field=_raw "(?:[^ \n]* ){6}(?P[\w]+)\b (?<=\B)"

Sample log line:
2018-11-28 20:03:17,998 [pool-3-thread-123:1.1.1.1:####-NN] INFO root - Poll (req)

"Poll" is the field I am looking for.

I'm confused about this problem because I have deleted the field extraction and redid it using the field extraction tool and it will show up for a brief time but then disappear. I tried the debug/refresh endpoint as well as | extract reload=true but it does not show up. Any suggestions where to look or what else to do would be great.

Thanks

[DavidHourani]

Hey there,

How about you give this regex a shot ?

EXTRACT-Action = (?:[^\b\n]+\b){6}(?<Action>\w+)\b

Cheers,
David

[mweissha]

Thanks David, that regex doesn't work to pull the field I need.

Either way, it's not the regex that is the problem, see my comments above for the resolution.

[DavidHourani]

good to know its working for you!

[woodcock · Accepted solution]

It is possible that another app is interfering with (overwriting/obliterating) Action. Use btool or disable apps one-by-one until it starts working, then see what is in the offending app.

[mweissha]

As this is a production server I can't just willy-nilly disable apps. I have taken it down to as few as is possible and I ran "btool check --debug" and fixed any errors that came up (nothing related to extracts or the search app that this extract resides in) Still no extract comes up. There are no extracts that are even close in name to this one.

However, when I create a brand new extraction with the name "FooBar" using the same regex it then shows up in results. This led me to dig further into anything that may be referencing my extracted field. I saw a field alias, which btw was in place before the upgrade, and I deleted that and lo-and-behold my extraction shows up. After a couple rounds of reloading and calling to the debug/refresh endpoint it still remains. I see now that it was the field alias "domain_action AS Action"

I wonder what changed between 6.5.3 -> 7.2 that would cause this behaviour? It's not as if "domain_action" was showing up in my results either but somehow it was not allowing "Action" to show up.

Thanks for the help in leading me to look around some more. I'll accept this as the answer with the caveat that it wasn't any specific app causing the problem but a field alias just so people in the future may know where to look.