Trying to only foward messages that pass a regex filter, and throw non-matchers away.

[srubik]

I have a heavy forwarder configured to send messages to a receiver. The receiver is able to receive all the messages. Now I'm trying to configure the forwarder to only send messages which match a regular expression. When I try, I get no messages being forwarded. In this simple example, I intend any message which contains "allow" to be forwarded, and the rest to be discarded. Any ideas why no messages are being forwarded?

inputs.conf

[monitor:///var/log/test2]
ignoreOlderThan = 14d

outputs.conf

[tcpout]
defaultGroup = logs-host.net_9997

[tcpout:logs-host.net_9997]
server = logs-host.net:9997

props.conf

[source::/var/log/test2/t2.log]
TRANSFORMS-set= setnull,setparsing

transforms.conf

[setnull]
REGEX = . 
DEST_KEY = queue
FORMAT = nullQueue

[setparsing] 
REGEX = .*allow.* 
DEST_KEY = _TCP_ROUTING 
FORMAT = logs-host.net_9997

[jonuwz]

You're throwing away the messages, and not including them again.

tcp forwarding occurs in the indexing queue, so you need to alter [setparsing] so that

DEST_KEY = queue

and

FORMAT = indexQueue

or

FORMAT = parsingQueue

Edit

You probably then need another transform called setrouting that sets the routing up afer setnull nad setparsing.