Getting Data In
Highlighted

INDEX_AND_FORWARD usage

Path Finder

I want to (index and) forward (to a syslog endpoint) some data that goes into a particular index on my indexer cluster.

These indexers mainly do not run inputs over and above the splunktcp://9997 listener as the data arrives at these indexers from universal forwarders and has in some cases already passed through a heavy forwarding layer, and some is direct to the indexers.

To begin with, I want to index all data and forward none (where I am now)
I want to change to indexing all data and also forwarding all data.
Then I want to index some data and forward all data (reducing what is indexed to a list of regex matches).

I don't quite know how to make sense of the INDEXANDFORWARD routing keys, they state I need to declare "INDEXANDFORWARDROUTING=local" for my inputs, but in most cases, my inputs are not local.

Also, are there any good examples of how to use INDEXANDFORWARD based on props/transforms matches ?

0 Karma
Highlighted
Highlighted

Re: INDEX_AND_FORWARD usage

Splunk Employee
Splunk Employee

Index and forward will not forward data in syslog format. It will forward it as Splunk Cooked data, meaning that only a Splunk HF / Indexer on the other end can process the feed.

If you truly want to forward to a 3rd party receiver, I'd look at the CEF App for Splunk (https://splunkbase.splunk.com/app/1847/). There is a custom command included that you can use to format and send messages in syslog format out of Splunk to a 3rd party receiver.

The other option is here : http://docs.splunk.com/Documentation/Splunk/6.5.3/Forwarding/Forwarddatatothird-partysystemsd

0 Karma
Highlighted

Re: INDEX_AND_FORWARD usage

Path Finder

One of the reasons we want to do this is that we don't want to index all the data that we've brought in using our UFs. As a result, I don't think the CEF app can help us.

We want to send some of a particular sourcetype to a syslog destination.
we want to
First - Index all of it AND forward all of it.
Second - Index a small amount of it AND forward all of it

I think this means we need some content based filtering and I currently have this setup:
outputs:
{code}
[syslog:Sendtosyslogdest]
type = tcp
server = syslog
server:10518
timestampformat = %Y-%m-%dT%H:%M:%S.%3N%z

[indexAndForward]
index = true
selectiveIndexing = true
{code}

props
{code}
[mysourcetype]
TRANSFORMS-filtering = 1-forward-all-data,6-Index-bits-of-it
{code}

transforms
{code}
[1-forward-all-data]
REGEX = .
DESTKEY = _SYSLOGROUTING
FORMAT = Sendtosyslog_dest

[6-Index-bits-of-it]
DESTKEY = _INDEXANDFORWARDROUTING
FORMAT = local
REGEX = (?msi)\"?(some|regex|matches|for|data)\"?
{code}

I've heard two things which make me question whether to continue on this tack.
1. That I shouldn't be trying to do this on an indexer, I should only be trying to do this on a 'heavy forwarder'
2. That I can only do this with splunktcp type 'cooked' data - as opposed to syslog format. In some cases (we have a HF layer in place) the data is arriving with us already parsed.

Having said that, this sort of seems to be working (I haven't done exhaustive testing).

Can someone answer the 2 questions at the end ?

Thanks

0 Karma