Sending uncooked data from indexer level

[gyarici]

Hi all,

I am sending data from intermediate forwarder to indexer and during indexing, I would like to send raw "uncooked data" to 3rd party application.
Recently I tried to use CEF app index and forward but , it is working but it is becoming cooked data.

Is there any way to handle this from indexer level?

Thanks

[codebuilder]

To accomplish the same data cloning with a universal forwarder, you can use the example below.

 outputs.conf:
 [tcpout]
 defaultGroup=Group1,Group2

 [tcpout:Group1]
 disabled=false
 sendCookedData=false
 server=10.x.xx.1:9997

 [tcpout:Group2]
 disabled=false
 sendCookedData=false
 server=10.x.xx.2:1234

[codebuilder]

This example should accomplish what you are trying to do. This combination of configs will send uncooked, cloned data, to different indexer groups (or individual indexers).

On your intermediate forwarder, configure the following under $SPLUNK_HOME/etc/system/local/:

props.conf:
[your:sourcetype:name]
TRANSFORMS-routing = cloneAll

transforms.conf:
[cloneAll]
REGEX=(.)
DEST_KEY=_TCP_ROUTING
FORMAT=Group1,Group2

outputs.conf:
[tcpout]
defaultGroup=nothing

[tcpout:Group1]
disabled=false
sendCookedData=false
server=10.x.xx.1:9997

[tcpout:Group2]
disabled=false
sendCookedData=false
server=10.x.xx.2:1234

Forgot to mention, cycle Splunk to implement the changes.

[gyarici]

Thanks for the answer. So, i need to send uncooked data from IF to indexer first (?) Anyhow, i need to send uncooked data directly from indexers to 3rd party app ! I think it should be performed before indexer parsing

[gcusello]

Hi @gyarici,
I had the same problem and I received this answer both from Community and Splunk Support:
if you have an intermediate Heavy Forwarder isn't possible to do this on Indexers.
But you can do the fork at Heavy Forwarder level.

Ciao.
Giuseppe

[gyarici]

Thanks for the information. I think if there is no way, this should be enhancement point as many of the architectures have tones of UF/IF and it is not practical/efficient to use UF to send the data to 3rd party apps

[codebuilder]

That's not true. Assuming you do not want or need to do any parsing at the forwarder level, you can accomplish this same task with a UF. See my second answer.

[gyarici]

Thanks alot for the response.

Ok. Then could you please clarify how I can configure indexers to send uncooked data to the 3rd party applications/servers?

Send uncooked data:

1.Intermediate Forwarder-> 2.Indexer -> 3.3rd party server

I assumed between step 1 and 2 above are ok with your configuration. I need to know more about between step and step 3 configuration which i should implement on indexers.

Thanks

[codebuilder]

Sending data directly from an indexer to a 3rd party system is not good practice nor a good idea. The supplied solutions are the accepted method to accomplish what you're trying to do.

[13tsavage]

I am not 100% sure if indexers can forward data the way you are requesting. This would mean that before indexers can parse and begin indexing the data, you want them to first send the uncooked data to other systems.