Index all but one input

claudio_manig — Tue, 29 Sep 2020 13:13:38 GMT

Guys-

I'm facing an (apparantely) challenging task:
I have a standalon splunk test instance which serves as a first point of ingestion for new inputs- however, what i want to achieve is the following:
As this instance runs on a windows server i want to have the configured os inputs (eventlog, perfmon) to be forwarded to our production instance and keep the rest local indexed.

What I have tried so far:

Easiest approach from my pov:
Created an outputs.conf

[tcpout:fwd_to_prod]
server = t800.skynet.net:9997

And a referring input in inputs.conf as follows:

[WinEventLog://Application]
_TCP_ROUTING = fwd_to_prod

Result: EVERYTHING is going to be forwarded to my production instance, including all internal stuff

After some research i tried it with the more complicated way, using a transforms to do so:
I had the same outputs.conf:

[tcpout:fwd_to_prod]
server = t800.skynet.net:9997

Added the following transforms stanza in transforms.conf:

#forward win events to prod splunk
[forward_prod]
DEST_KEY = _TCP_ROUTING
FORMAT = fwd_to_prod
REGEX = .

Then i referred to my transforms in my props.conf:

[WinEventLog://Application]
TRANSFORMS-App = forward_prod

Result: EVERYTHING is going to be forwarded to my production instance, including all internal stuff

Its weird somehow -
Yes i did read the docs, and yes, i'm aware that forwardandIndex and selectiveIndex is a way but it the docs all refer to the other way round, indexing one type and forward everything else! I don't want to set "_INDEX_AND_FORWARD_ROUTING" on all my inputs expect the windows one as you might understand.

I even tried to add a dummy tcp output group as default group in my outputs.conf with no effect - adding the "localhost" as the default target group resulted in no forwarding or indexing at all.

Any help is appreciated

Cheers

Re: Index all but one input

somesoni2 — Tue, 29 Sep 2020 13:13:41 GMT

Something like this should work

etc/system/local/outputs.conf (configure selective indexing)

[indexAndForward]
index=true
selectiveIndexing=true 

[tcpout:fwd_to_prod]
server = yourIndexer:9997

etc/system/local/inputs.conf (configure default behaviour as local indexing for all data inputs)

[default]
_INDEX_AND_FORWARD_ROUTING = MySecKey

Add the _TCP_ROUTING attribute to the stanzas of each input that you want to forward. E.g.

[WinEventLog://Application]
_TCP_ROUTING=fwd_to_prod

Re: Index all but one input

claudio_manig — Tue, 14 Mar 2017 14:45:18 GMT

Oh I see i missed the [default] option in inputs.conf to make it global - worked like a charm, thanks a lot!
Can you do me another favour and use my first hostname on your answer - thx

topic Index all but one input in Getting Data In

Index all but one input

Re: Index all but one input

Re: Index all but one input