How do I filter out DEBUG entries from a linux / Unix logfile with the heavy forwarder?

[gozulin]

We're having some licensing violations when we need to turn on DEBUG on some of our services and we'd like to just have a regex nullqueue any debug entries before forwarding them to the indexers.

[mukherjee_mk]

Thanks for your help folks. I notice that we have to keep these rows in the right order though. The name of the sourcetype should be at the beginning of the segment.

in props.conf (notice that the sourcetype is the first line of the segment):
[mysourcetype]
TRANSFORMS-null= setnull
NO_BINARY_CHECK = 1
pulldown_type = 1

In transforms.conf:
[setnull]
REGEX = DEBUG
DEST_KEY = queue
FORMAT = nullQueue

[gozulin · Accepted solution]

Using the previous answer, here is what worked to filter out DEBUG messages:

in props.conf:

TRANSFORMS-null= setnull
[mysourcetype]
NO_BINARY_CHECK = 1
pulldown_type = 1

In transforms.conf:

[setnull]
REGEX = [DEBUG]
DEST_KEY = queue
FORMAT = nullQueue

[gozulin]

Huh, you're right of course. It's weird because the content of the file actually has backslashes in it. Not sure why they didn't show up!

[setnull]
REGEX = \[DEBUG\]
DEST_KEY = queue
FORMAT = nullQueue

[David]

Have you verified that this is not matching more than you intended? In regex terms, that should match anything with a capital D, E, B, U, or G.

[David]

You should be able to follow the guidance of this answers post but replace the regex with DEBUG. You could make the regex more specific by providing a few example logs (e.g., LogLevel DEBUG if that's what your logs look like).