Getting Data In

How to fix my universal forwarder configurations so that Splunk only forwards the data I want to monitor to a third-party system?

anton085
Path Finder

I am trying to forward to a third-party system from a Universal forwarder. I have tried two approaches. In both cases I am receiving a lot of unnecessary data on the third-party end. It looks like Splunk is not only forwarding the file that I am monitoring but also internal logs as well. What can I do to fix this? I am attaching conf files for both:

Approach 1: use props, transforms, and outputs

props.conf
[source::/home/abc/splunk-test/test.txt]
TRANSFORMS-routing=monitoring

transforms.conf
[monitoring]
REGEX=.
DEST_KEY=_TCP_ROUTING
FORMAT=monitoring_tcp

outputs.conf
[tcpout]
defaultGroup=group_a

[tcpout:group_a]
disabled=true

[tcpout:monitoring_tcp]
sendCookedData=false
server=x.x.x.x:514

Approach 2: use inputs.conf and outputs.conf (I deleted everything from props and transforms)
inputs.conf
[default]
host=abc

[monitor:///home/abc/splunk-test/test.txt]
_TCP_ROUTING=monitoring_tcp

outputs.conf
[tcpout]
defaultGroup=group_a

[tcpout:group_a]
disabled=true

[tcpout:monitoring_tcp]
sendCookedData=false
server=x.x.x.x:514

0 Karma
1 Solution

anton085
Path Finder

I have figured out two ways to block internal logs from being forwarded:

  1. inputs.conf
    use disabled=true for inputs that have to be blocked. For me, I blocked these ones, which had _TCP_ROUTING=* set in the default inputs.conf files inside $SPLUNK_HOME/etc/system/default and $SPLUNK_HOME/etc/apps/SplunkUniversalForwarder/default

    [monitor://$SPLUNK_HOME/var/log/splunk]
    disabled=true
    [monitor://$SPLUNK_HOME/var/log/splunk]
    disabled=true
    [monitor://$SPLUNK_HOME/var/log/splunk/splunkd.log]
    disabled=true
    [monitor://$SPLUNK_HOM/var/log/splunk/metrics.log]
    disabled=true

  2. outputs.conf
    use the blacklist. According to the documentation, a whitelist can override a blacklist when both have the same number, and the filtering order is based on increasing number. Also, the filtering will only work under the [tcpout] stanza.
    So, here, the whitelist would win

    forwardedindex.0.whitelist
    forwardedindex.0.blacklist

And here, the blacklist would win

forwardedindex.0.whitelist
forwardedindex.1.blacklist

From the default conf files, I figured out that there are 3 lists going from 0 to 2. So I added the following snippet in $SPLUNK_HOME/etc/system/local/outputs.conf, and it worked.

[tcpout]
forwardedindex.3.blacklist = (_internal|_audit|_telemetry|_introspection)

View solution in original post

anton085
Path Finder

I have figured out two ways to block internal logs from being forwarded:

  1. inputs.conf
    use disabled=true for inputs that have to be blocked. For me, I blocked these ones, which had _TCP_ROUTING=* set in the default inputs.conf files inside $SPLUNK_HOME/etc/system/default and $SPLUNK_HOME/etc/apps/SplunkUniversalForwarder/default

    [monitor://$SPLUNK_HOME/var/log/splunk]
    disabled=true
    [monitor://$SPLUNK_HOME/var/log/splunk]
    disabled=true
    [monitor://$SPLUNK_HOME/var/log/splunk/splunkd.log]
    disabled=true
    [monitor://$SPLUNK_HOM/var/log/splunk/metrics.log]
    disabled=true

  2. outputs.conf
    use the blacklist. According to the documentation, a whitelist can override a blacklist when both have the same number, and the filtering order is based on increasing number. Also, the filtering will only work under the [tcpout] stanza.
    So, here, the whitelist would win

    forwardedindex.0.whitelist
    forwardedindex.0.blacklist

And here, the blacklist would win

forwardedindex.0.whitelist
forwardedindex.1.blacklist

From the default conf files, I figured out that there are 3 lists going from 0 to 2. So I added the following snippet in $SPLUNK_HOME/etc/system/local/outputs.conf, and it worked.

[tcpout]
forwardedindex.3.blacklist = (_internal|_audit|_telemetry|_introspection)

kutzi
Path Finder

I tried method 1, but it doesn't seem to work.
I put

[monitor://$SPLUNK_HOME/var/log/splunk/metrics.log]
disabled=true

into $SPLUNK_HOMe/etc/system/local/inputs.conf

Did you put your inputs.conf somewhere else?

0 Karma

sbbadri
Motivator

[tcpout:monitoring_tcp]
sendCookedData=false
server=x.x.x.x:514
forwardedindex.0.blacklist = (_internal|_audit)

please go through below link for more details,

http://docs.splunk.com/Documentation/Splunk/6.6.2/Forwarding/Routeandfilterdatad

0 Karma

anton085
Path Finder

I have tried this and it didn't work. Moreover, the link said to add blacklist under a global tcp stanza only, I tried that and that didn't work either.

0 Karma

anton085
Path Finder

actually added the following lines in outputs.conf but to no avail:

[tcpout]
forwardedindex.0.blacklist = (_internal|_audit|_telemetry|_introspection)
forwardedindex.2.blacklist = (_internal|_audit|_telemetry|_introspection)

0 Karma
Get Updates on the Splunk Community!

Data Preparation Made Easy: SPL2 for Edge Processor

By now, you may have heard the exciting news that Edge Processor, the easy-to-use Splunk data preparation tool ...

Introducing Edge Processor: Next Gen Data Transformation

We get it - not only can it take a lot of time, money and resources to get data into Splunk, but it also takes ...

Tips & Tricks When Using Ingest Actions

Tune in to learn about:Large scale architecture when using Ingest ActionsRegEx performance considerations ...