Is forwardedindex.. only applicable for TCP connection?

[cemiam]

Hi,

I am using heavy forwarder to forward syslogs to a 3rd party syslog aggregator. I am trying to filter some of the forwarded events on heavy forwarder and noticed that it is already sending audit logs even if I blacklist all internal indexes.

According to below outputs.conf it should not forward audit logs. After checking outputs.conf guide I have noticed forwardedindex..whitelist = and forwardedindex..blacklist = are only applicable under the global [tcpout] stanza. As it is an TCP based stanza I think it is not possible to filter UDP events. I need to make some additional regex-based filterings but I don't think it will not be possible. Is there any way to do this?

outputs.conf

[tcpout]
maxQueueSize = auto
forwardedindex.0.whitelist = .*
forwardedindex.1.blacklist = _.*
forwardedindex.2.blacklist = (_audit|_introspection|_telemetry)
forwardedindex.filter.disable = false

[syslog]
defaultGroup = syslogGroup
forwardedindex.0.whitelist = .*
forwardedindex.1.blacklist = _.*
forwardedindex.2.blacklist = (_audit|_introspection|_telemetry)
forwardedindex.filter.disable = false

[syslog:syslogGroup]
server = 10.xx.x.xxx:514
sendCookedData = false

[syslog-server://10.xx.x.xxx:514]

[tcpout]
defaultGroup =
indexAndForward = 1

[jkat54 · Accepted solution]

Are you sure other outputs are not overriding your blacklist settings?

Try this to see if another app is overriding:

./splunk btool outputs list —debug

I’m not sure why you have indexAndForward enabled, you didn’t mention keeping the data on the heavy forwarder so I’m not sure if you want to do that or not.

As for applying your transformations to UDP data, yes you can do that.

See this documentation:
http://docs.splunk.com/Documentation/Splunk/7.0.3/Forwarding/Routeandfilterdatad

Note you will be using _SYSLOG_ROUTING instead of _TCP_ROUTING. There’s also _HTTP_ROUTING according to the link.

[cemiam]

Thanks for the reply. I have enabled indexandForward just to troubleshooting. I have disabled it. You can find the btool commands output below. I couldn't find any overriding.

/opt/splunk/etc/apps/SplunkForwarder/local/outputs.conf [syslog]
/opt/splunk/etc/system/local/outputs.conf defaultGroup = syslogGroup
/opt/splunk/etc/system/local/outputs.conf forwardedindex.0.whitelist = .*
/opt/splunk/etc/system/local/outputs.conf forwardedindex.1.blacklist = _.*
/opt/splunk/etc/apps/SplunkForwarder/local/outputs.conf forwardedindex.2.blacklist = (_audit|_introspection|_telemetry)
/opt/splunk/etc/system/local/outputs.conf forwardedindex.filter.disable = false
/opt/splunk/etc/system/default/outputs.conf maxEventSize = 1024
/opt/splunk/etc/system/default/outputs.conf priority = <13>
/opt/splunk/etc/system/local/outputs.conf sendCookedData = false
/opt/splunk/etc/system/default/outputs.conf type = udp
/opt/splunk/etc/system/local/outputs.conf [syslog-server://10.xx.x.xxx:514]
/opt/splunk/etc/system/local/outputs.conf [syslog:syslogGroup]
/opt/splunk/etc/system/local/outputs.conf server = 10.xx.x.xxx:514
/opt/splunk/etc/apps/SplunkForwarder/local/outputs.conf [tcpout]
/opt/splunk/etc/system/default/outputs.conf ackTimeoutOnShutdown = 30
/opt/splunk/etc/system/default/outputs.conf autoLBFrequency = 30
/opt/splunk/etc/system/default/outputs.conf autoLBVolume = 0
/opt/splunk/etc/system/default/outputs.conf blockOnCloning = true
/opt/splunk/etc/system/default/outputs.conf blockWarnThreshold = 100
/opt/splunk/etc/system/default/outputs.conf cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:AES256-GCM-SHA384:AES128-GCM-SHA256:AES128-SHA256:ECDH-ECDSA-AES256-GCM-SHA384:ECDH-ECDSA-AES128-GCM-SHA256:ECDH-ECDSA-AES256-SHA384:ECDH-ECDSA-AES128-SHA256
/opt/splunk/etc/system/default/outputs.conf compressed = false
/opt/splunk/etc/system/default/outputs.conf connectionTimeout = 20
/opt/splunk/etc/system/local/outputs.conf defaultGroup =
/opt/splunk/etc/system/default/outputs.conf disabled = false
/opt/splunk/etc/system/default/outputs.conf dropClonedEventsOnQueueFull = 5
/opt/splunk/etc/system/default/outputs.conf dropEventsOnQueueFull = -1
/opt/splunk/etc/system/default/outputs.conf ecdhCurves = prime256v1, secp384r1, secp521r1
/opt/splunk/etc/system/default/outputs.conf forceTimebasedAutoLB = false
/opt/splunk/etc/system/local/outputs.conf forwardedindex.0.whitelist = .*
/opt/splunk/etc/system/local/outputs.conf forwardedindex.1.blacklist = _.*
/opt/splunk/etc/apps/SplunkForwarder/local/outputs.conf forwardedindex.2.blacklist = (_audit|_introspection|_telemetry)
/opt/splunk/etc/apps/SplunkForwarder/default/outputs.conf forwardedindex.2.whitelist = (_audit|_introspection|_telemetry)
/opt/splunk/etc/system/local/outputs.conf forwardedindex.filter.disable = false
/opt/splunk/etc/system/default/outputs.conf heartbeatFrequency = 30
/opt/splunk/etc/system/local/outputs.conf indexAndForward = 0
/opt/splunk/etc/system/default/outputs.conf maxConnectionsPerIndexer = 2
/opt/splunk/etc/system/default/outputs.conf maxFailuresPerInterval = 2
/opt/splunk/etc/system/local/outputs.conf maxQueueSize = auto
/opt/splunk/etc/system/default/outputs.conf readTimeout = 300
/opt/splunk/etc/system/default/outputs.conf secsInFailureInterval = 1
/opt/splunk/etc/system/local/outputs.conf sendCookedData = false
/opt/splunk/etc/system/default/outputs.conf sslQuietShutdown = false
/opt/splunk/etc/system/default/outputs.conf sslVersions = tls1.2
/opt/splunk/etc/system/default/outputs.conf tcpSendBufSz = 0
/opt/splunk/etc/system/default/outputs.conf useACK = false
/opt/splunk/etc/system/default/outputs.conf writeTimeout = 300
[root@heavyforwarder bin]#

For using transforms.conf I think I should index data. This is not something I should do. I have enabled indexandForward just for troubleshooting purposes.

[cemiam]

Somehow it doesn't allow me to upload inputs.conf output

[cemiam]

I am not sure what is wrong with the post but although I have enough characters left it doesn't allow me to upload inputs.con btool command output. You can find the my inputs.conf output under /local directory.

inputs.conf

[default]
host = heavyforwarder.dataserv.local

[udp://515]
sourcetype = syslog
disabled = 0

[jkat54]

Sorry I meant props.conf and transforms.conf related to this input.

[cemiam]

I have tried to send btool outputs but the character limitation doesn't allow me to do it. I have re-checked the props.conf and transforms.conf. Both are on the default. As I am planning send this data directly to a 3rd party I didn't make any configuration.

[jkat54]

Ok, so you haven’t done what the documentation says to do. See if this document is easier to follow:

http://docs.splunk.com/Documentation/Splunk/7.0.3/Forwarding/Forwarddatatothird-partysystemsd

[cemiam]

Hi,

I have just followed this document and now I am able to filter specific event with the help of regex. The problem is I am still having trouble to blacklisting audit logs. As they are not in common format it is hard to filter audit logs with regex.

[jkat54]

Did you try a regex that matches the index name?

[cemiam]

According guide regex helps to filter data but I think it can be filtered via hostname on props.conf.

[jkat54]

Yeah you can do that.

[jkat54]

Can you post the inputs.conf too?