Routing and filtering is not working as expected

[pankajupadhyay]

Hi,

I was trying to achieve that particular sourcetype logs should reach to target 1 and not to target 2.

Even i tried to send particular logs event to target 1 and rest of the event should be discard but it is not working at all.

Below configuration is on heavy forwarder.

inputs.conf

[tcp://1514]
sourcetype = syslog
connection_host = dns

_TCP_ROUTING=target1-----------When transforms.conf was not working then i define the tcp routing in inputs.conf then it stopped sending the logs to target2 but filtering of the event is not working.

In tranforms.conf 

[vmwarelogs]
REGEX=(logged out|Rejected password for user|Cannot login|logged in as|Accepted user for user|was updated on host|Password was changed for account|Destroy VM called)
DEST_KEY=_TCP_ROUTING
FORMAT=target1

[discarlogs]

REGEX = .

DEST_KEY = queue

FORMAT = nullQueue

In props.conf

[vmw-syslog]

Tranforms-routing=vmwarelogs,discarlogs

Outputs.conf 

target1 

target2---- defualt group

1. Issue number 1 : when i have defined the target group in tranforms.conf to send the logs to target 1 and not to target 2 but still it target 2 was getting the llogs 

Then i have define in inputs.conf itselft then i achived the first objective just to forward the logs to target1 but in this case inputs.conf will take the precedence and transforms.conf filter is not working.

_TCP_ROUTING=target1-----------When transforms.conf was not working then i define the tcp routing in

inputs.conf then it stopped sending the logs to target2 but filtering of the event is not working.

Issue 2: I wanted to send the specific logs to target1 and rest of the event need to be discard and if calling the nullquue configuraiton in props.conf then it is not sending the logs at all.

Please help me out how i can achive my objective.

Is that possible to whitelist attribute in input.conf itself ??? 

[gbeatty]

[vmw-syslog]
Tranforms-routing=vmwarelogs,discarlogs

[vmwarelogs]
REGEX=(logged out|Rejected password for user|Cannot login|logged in as|Accepted user for user|was updated on host|Password was changed for account|Destroy VM called)
DEST_KEY=_TCP_ROUTING
FORMAT=target1

[discarlogs]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

I have not been able to test on my own, but I believe your problem lies with trying to apply both transforms with one line of props.  It is possible for your events to match on both, so the the events are sent to the null queue, overriding the assignment to TCP ROUTING.  I would try reversing the order as @saravanan90 suggested, but also reducing the complexity of your [vmwarelogs] regex to one kind of event, until you have the routing down, then expand it to cover the other events.

The example given in Splunk documentation supports this order.

Props
[source::/var/log/messages]
TRANSFORMS-set= setnull,setparsing

Transforms
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \[sshd\]
DEST_KEY = queue
FORMAT = indexQueue

https://docs.splunk.com/Documentation/Splunk/8.1.2/Forwarding/Routeandfilterdatad#Filter_event_data_...

Lastly, check to make sure that the data is set to go through the parsing queue whereever you are attempting to apply props.

[saravanan90]

Can you try changing the below.. 

In props.conf

[vmw-syslog]

Tranforms-routing=discarlog,vmwarelogs

[pankajupadhyay]

@saravanan90  I tried but it is not working at all even tac is not able to give the solution.

I tried all the possible way.