Getting Data In

How do I edit my single-machine deployments outputs.conf to send out data for only 1 index?

Federica_92
Communicator

Hi everyone,

I'm trying to use splunk as heavy forwarder to send out only 1 index, but it doesn't work. Could someone please help me? I think there is something wrong in the outputs.conf.

[tcpout]
deafultGroup = nothing

[tcpout:alerts]
server = 10.28.100.121:9998
indexAndForward = 1

[tcpout:alerts]
indexAndForward = 1
#Forward data for the "alerts" index
forwardedindex.0.whitelist = alerts

transforms.conf:

[alerts]
REGEX = .
DEST_KEY=_TCP_ROUTING
FORMAT=alerts

props.conf

[index::alerts]
TRANSFORMS-routing = alerts

Thanks,
Federica

0 Karma

lguinn2
Legend

First, your outputs.conf is wrong. It has misspellings and as has been pointed out, you have duplicate stanzas. Plus, most of the settings must appear in the tcpout stanza at the global level.

[tcpout]
defaultGroup = alerts
indexAndForward = true
forwardedindex.0.whitelist = alerts

[indexAndForward]
index = true
selectiveIndexing = false

 [tcpout:alerts]
server = 10.28.100.121:9998

Your props.conf has no effect, because there is no such stanza as [index::xyz]. Therefore, your transforms.conf was never invoked, which is just as well because the regular expression in the REGEX would not have filtered anything. Just remove the props.conf and transforms.conf entries; you don't need them.

I am not sure that the whitelist is going to work. If it does not, then replace it with these two lines instead:

forwardedindex.0.blacklist = *
forwardedindex.1.whitelist = alerts

In the future, you might want to run

splunk btool check

from the command line, which may identify syntax errors in your configuration files. btool can't catch everything, but it can help.

0 Karma

Federica_92
Communicator

I tried, with both, whitelist only and whitelist and blacklist.
I got the same issue as before, the traffic is being sent, but not filtered...

0 Karma

stmyers7941
Path Finder

I believe you need a single stanza for [tcpout:alerts] with all the settings underneath it. With 2 stanzas, It's going to use the latest and ignore the former.

[tcpout]
 deafultGroup = nothing
[tcpout:alerts]
server = 10.28.100.121:9998
indexAndForward = 1
#Forward data for the "alerts" index
forwardedindex.0.whitelist = alerts
0 Karma

Federica_92
Communicator

No, I already tried, doesn't filter the data. It sends everything.

0 Karma

stmyers7941
Path Finder

Have you tried Selective Indexing?

#
# Perform selective indexing and forwarding
#
# With a heavy forwarder only, you can index and store data locally, as well as
# forward the data onwards to a receiving indexer. There are two ways to do
# this:

# 1. In outputs.conf:
[tcpout]
defaultGroup = indexers

[indexAndForward]
index=true
selectiveIndexing=true

[tcpout:indexers]
server = 10.1.1.197:9997, 10.1.1.200:9997

# 2. In inputs.conf, Add _INDEX_AND_FORWARD_ROUTING for any data that you want
#    index locally, and
_TCP_ROUTING=<target_group> for data to be forwarded.

[monitor:///var/log/messages/]
_INDEX_AND_FORWARD_ROUTING=local

[monitor:///var/log/httpd/]
_TCP_ROUTING=indexers
0 Karma

Federica_92
Communicator

I can't monitor, because the file aren't stored anywhere, I need to send out data that splunk is producing and indexing. The only way to access directly to the data, it's to access to the kv store

0 Karma

stmyers7941
Path Finder

What's your input on the heavy forwarder in question?

0 Karma

Federica_92
Communicator

I have an alert manager ( the app) that is executing a script, whenever an alerts is being triggered. At that stage it produce some data, using the rest API that are saved on the kv stores

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.