Getting Data In

Sending uncooked data from indexer level

gyarici
Path Finder

Hi all,

I am sending data from intermediate forwarder to indexer and during indexing, I would like to send raw "uncooked data" to 3rd party application.
Recently I tried to use CEF app index and forward but , it is working but it is becoming cooked data.

Is there any way to handle this from indexer level?

Thanks

0 Karma

codebuilder
Influencer

To accomplish the same data cloning with a universal forwarder, you can use the example below.

 outputs.conf:
 [tcpout]
 defaultGroup=Group1,Group2

 [tcpout:Group1]
 disabled=false
 sendCookedData=false
 server=10.x.xx.1:9997

 [tcpout:Group2]
 disabled=false
 sendCookedData=false
 server=10.x.xx.2:1234
----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

codebuilder
Influencer

This example should accomplish what you are trying to do. This combination of configs will send uncooked, cloned data, to different indexer groups (or individual indexers).

On your intermediate forwarder, configure the following under $SPLUNK_HOME/etc/system/local/:

props.conf:
[your:sourcetype:name]
TRANSFORMS-routing = cloneAll

transforms.conf:
[cloneAll]
REGEX=(.)
DEST_KEY=_TCP_ROUTING
FORMAT=Group1,Group2

outputs.conf:
[tcpout]
defaultGroup=nothing

[tcpout:Group1]
disabled=false
sendCookedData=false
server=10.x.xx.1:9997

[tcpout:Group2]
disabled=false
sendCookedData=false
server=10.x.xx.2:1234

Forgot to mention, cycle Splunk to implement the changes.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

gyarici
Path Finder

Thanks for the answer. So, i need to send uncooked data from IF to indexer first (?) Anyhow, i need to send uncooked data directly from indexers to 3rd party app ! I think it should be performed before indexer parsing

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @gyarici,
I had the same problem and I received this answer both from Community and Splunk Support:
if you have an intermediate Heavy Forwarder isn't possible to do this on Indexers.
But you can do the fork at Heavy Forwarder level.

Ciao.
Giuseppe

0 Karma

gyarici
Path Finder

Thanks for the information. I think if there is no way, this should be enhancement point as many of the architectures have tones of UF/IF and it is not practical/efficient to use UF to send the data to 3rd party apps

0 Karma

codebuilder
Influencer

That's not true. Assuming you do not want or need to do any parsing at the forwarder level, you can accomplish this same task with a UF. See my second answer.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

gyarici
Path Finder

Thanks alot for the response.

Ok. Then could you please clarify how I can configure indexers to send uncooked data to the 3rd party applications/servers?

Send uncooked data:

1.Intermediate Forwarder-> 2.Indexer -> 3.3rd party server

I assumed between step 1 and 2 above are ok with your configuration. I need to know more about between step and step 3 configuration which i should implement on indexers.

Thanks

0 Karma

codebuilder
Influencer

Sending data directly from an indexer to a 3rd party system is not good practice nor a good idea. The supplied solutions are the accepted method to accomplish what you're trying to do.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

13tsavage
Communicator

I am not 100% sure if indexers can forward data the way you are requesting. This would mean that before indexers can parse and begin indexing the data, you want them to first send the uncooked data to other systems.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...