Can you index a certain sourcetype and forward the remaining?

[mustafag1]

Hi
I am new to Splunk and am trying to forward a specific sourcetype of data out. That part is successful but now I am having trouble with the next part; indexing the remaining sourcetypes.
I am using a Windows Universal Forwarder to forward all logs to a Splunk Enterprise Instance. I want to Index the Perfmon logs but forward the Security and Application logs to a third Party source. How can I achieve this?
So far all the documentation seems to indicate using Selective Indexing but there the information suggests setting the entire log to either be indexed or forwarded or both, not just the specific sourcetype.

[mustafag1 · Accepted solution]

Hi

I seem to have found what I required from https://answers.splunk.com/answers/614054/how-to-index-all-locally-and-forward-specific-sour.html

Here we have work around to changing inputs.conf which did not allow us to filter sourcetypes. We are still using Selective Forwarding except now we can divert any specific sourcetype by identifying it in props.conf and giving it a transform in the stanza

[<sourcetype>]
TRANSFORMS-toindex = selectiveIndex

And we define this Transform in transforms.conf

[selectiveIndex]
REGEX = .
DEST_KEY = _INDEX_AND_FORWARD_ROUTING
FORMAT = local

We also add a stanza to transforms.conf

[accepted_keys]
is_valid = _INDEX_AND_FORWARD_ROUTING

This takes away the invalid key error from the selectedIndex stanza. Now we are Indexing that sourcetype only. The ones that need to be forwarded can be defined as usual when forwarding to 3rd party Indexers (https://docs.splunk.com/Documentation/Splunk/7.3.1/Forwarding/Forwarddatatothird-partysystemsd).

Since we are using selective Indexing we need to add a stanza to outputs.conf as well

[indexAndForward]
index = true
selectiveIndexing = true

[gcusello]

Hi mustafag1,
in the documentation, you can find instruction about how to forward data to a third party system https://docs.splunk.com/Documentation/Splunk/7.3.1/Forwarding/Forwarddatatothird-partysystemsd selecting a sourcetype, e.g.:
In props.conf on Universal Forwarder:

[wineventlog:Security]
TRANSFORMS-sec = security

In transforms.conf on Universal Forwarder:

[security]
REGEX = .
DEST_KEY=_TCP_ROUTING
FORMAT=third_party

in outputs.conf on Universal Forwarder:

[tcpout:third_party]
server=1.1.1.1:7999
sendCookedData=false

then you can find instructions about how to filter data on indexers at https://docs.splunk.com/Documentation/Splunk/7.3.1/Forwarding/Routeandfilterdatad :
In props.conf on Indexers:

[wineventlog:Security]
TRANSFORMS-null= setnull

In transforms.conf on Indexers:

[setnull]
REGEX = \[sshd\]
DEST_KEY = queue
FORMAT = nullQueue

Bye.
Giuseppe

[mustafag1]

Hello Giuseppe,

Thank you for answering my question. As per your suggestion I am adding two transforms to [Wineventlog:Security]

In props.conf on Splunk Enterprise Server:

[wineventlog:Security]
TRANSFORMS-null= setnull
TRANSFORMS-routing=send_to_syslog

And in transforms.conf I have two stanzas [send_to_syslog] and [setnull]. Can you please confrim that this is a valid operation? So far it seems that Splunk is no longer Indexing any security events (No security events on the Web GUI) but also it is not forwarding any events to third party system.

Also Note for clarity:
We do not want to alter the Forwarders. The idea is that we change the configurations on the Indexer so that it handles the indexing for most logs and forwards the logs with relevant sourcetypes, without indexing, to a third party. This way we do not have to modify the series of forwarders that may be sending logs to a main Splunk Indexer.

Mustafa

[mustafag1]

Also it seems that that the REGEX = [sshd] is designed for sshd events (as written in the documentation). If we set it to "." to apply to all events, nothing is indexed for that transform, but this also removes all forwarding.
Is this designed to for both forwarding and discarding (what I require) or is it a one or the other case (which it seems to be).