topic Re: Selective Filtered Indexing and Forwarding to 3rd party syslog in Getting Data In

Selective Filtered Indexing and Forwarding to 3rd party syslog

archme — Wed, 30 Sep 2020 03:32:29 GMT

Hello,

Our setup is as follows:

Windows/Unix UF -> HF -> IDX Clusters

Currently we are sending everything to IDX cluster and 1 copy of the logs to a 3rd party syslog server from the HF.

What we are trying to achieve is to send everything to the 3rd party syslog server and only send filtered logs to the idx clusters.

Given the scenario of the following sourcetype:
a) wineventlog
b) linux_secure
c) cisco:asa

I am trying to figure out how we can :
a) Send everything to the 3rd party syslog
b) Drop events with the eventcode 4624 for example in wineventlog from being send to indexer.
c) Only send events with "ssh" keyword for example in linux_secure to the indexer. Other events in linux_secure sourcetype should not be sent to the indexer.
d) send all cisco:asa events to the indexer.

Re: Selective Filtered Indexing and Forwarding to 3rd party syslog

mydog8it — Tue, 07 Jan 2020 20:51:59 GMT

Event filtering is one of the primary reasons to use an HF as an intermediate forwarder. Your license is not impacted until data reaches the IDX. Why not blacklist the undesired traffic there?

Re: Selective Filtered Indexing and Forwarding to 3rd party syslog

archme — Wed, 08 Jan 2020 12:04:48 GMT

I am thinking of putting the following on the HF

outputs.conf

[tcpout]
defaultGroup = primary_indexers

### this setting will send all the logs to third party syslog server. This setting seems to be working fine.

 [syslog]
 defaultGroup = EverythingtoSyslog
 [syslog:EverythingtoSyslog]
 #sendCookedData=false
 server=1.2.3.4:514
 type=udp
 maxEventSize=5096

props.conf

# For wineventlog sourcetype , i tried to put the following plus the transforms below..but it did not drop the event.

[wineventlog]
TRANSFORMS-set=wineventlog-setnull,wineventlog-setparsing

transforms.conf

### this following setting should drop  events with the eventcode 4624 for example in wineventlog from being send to indexer. The rest of the logs should be sent to primary_indexers. This should not conflict with linux_secure or cisco filtering

# For wineventlog sourcetype , i tried to put the following plus the props above..but it did not drop the event.  

[wineventlog-setparsing]
REGEX = .
DEST_KEY = queue
FORMAT =  indexQueue

[wineventlog-setnull]
REGEX=EventCode=(4624)
DEST_KEY = queue
FORMAT = nullQueue

### the following setting should Only send events with "ssh" keyword for example in linux_secure to the indexer. Other events in linux_secure sourcetype should not be sent to the indexer. This should not conflict with the wineventlog or cisco filtering.

not sure what to put here

### This should send all cisco:asa events to the indexer. This should not conflict with the above 2 settings.

not sure what to put here