Im a noobja not a ninja. I have a Windows based Splunk Enterprise single node index running 7.0.2. I'm trying to use it as a relay of sorts, I have a dataset coming into dedicated index, and I'd like to either forward that data or mirror the data to a 3rd party. There is no requirement to keep the data in splunk otherwise. I've read around quite a bit and I'm probably close, but I can't seem to get something right - so far I've only been successful at redirecting all the data to 3rd party, not a subset of data as preferred. When I apply my settings, I no longer see data real time in my splunk environment, but I do see data at the 3rd party endpoint.
I'm confused if I can use the index itself as a heavy forwarder, I didn't find a props.conf file so I created one in C:\Program Files\Splunk\etc\system\local.
-outputs.conf- [tcpout] defaultGroup=nothing [tcpout:3rdPartyDest] server=aaa.bbb.ccc.ddd:514 type=tcp sendCookedData=false -props.conf- [source] TRANSFORMS-routing = transforms_3rdParty -transforms.conf- [SiteCode] (already existed) filename = SiteCode.csv [transforms_3rdParty] REGEX=. DEST_KEY=_TCP_ROUTING FORMAT=3rdPartyDest
I've seen reference to indexAndForward flags as well as setting the output default group to nothing, but I can't seem to get the right combo working properly and don't want to redirect our flow via trial and error anymore.
Any help is appreciated!
If you want to clone data on your splunk and 3rd party splunk, then you can use following outputs.conf configurations
What if the 3rd party is not a Splunk instance, but a system capable of processing inbound syslog from Splunk.
I know we can forward newly indexed data from Splunk to such a 3rd party system, but can we replicate historical Splunk indexed data?
I replied above, I am successfully forwarding to a separate syslog (non splunk) device but I know there are some formatting issues, so I'm not entirely confident that my implementation is forwarding true Syslog formatted data. I think if you process (cook) it then it can re-send it in a different format. my intention was to not process too much of it, and ideally I didn't need it stored on splunk, I was just using it as a means to get the data from 3rd party to the other syslog collector that's not capable of natively leveraging the API from the 3rd party itself.
I don't know about forwarding the historical data you have already, my process relies on forwarding the data it is ingesting in real time.
This post is not stale, I have been working diligently to try and find the solution on my own...
The linked article is similar to other articles I have found and tried in the past.
I was not able to properly adapt the linked article to do what I need, and instead redirected all the logs to the device again.
I have a single splunk node/instance, which is receiving a multitude of data points into many separate indexes. There are no other heavy forwarders in the environment. I want to take data that is being ingested into a dedicated index and either replicate it, or forward it entirely (I don't care that splunk has the data) to a 3rd party device on 514. This data is not Syslog formatted, so I will be using the sendCookedData=false flag.
Ultimately, we have a need to monitor security logs from a product that is cloud based app. There is a dedicated SplunkApp to download this data from the cloud into our prem which is installed and operating properly. I need to get this data to a non-splunk SIEM, and the device can't reach out to the Cloud itself, so I am trying to use Splunk as a relay of sorts.
Any help is appreciated.
I did get it working, although it's also forwarding information from other areas of splunk I don't need so it's far form a perfect implementation. As a result I am dropping unneeded data at the other side, but in my case it's not causing any harm. Hopefully between the 3 conf files below you can put together what I am doing, I redacted the content.
In this implementation, I am downloading data from a 3rd party's API via the appropriate Splunk App (getting it into Splunk wasn't the hard part), I believe I am not processing it,or storing it locally, and then forwarding it off to a separate non splunk log collector (via syslog port, but maybe not in full syslog format) for additional analysis.
[tcpout:name of 3rd party]
filename = SiteCode.csv I THINK THIS IS IRRELEVANT, PREEXISTING
FORMAT=*_name of 3rd party*