Hi All,
Thank you for the assistance so far.
I just want to confirm my understanding and ask a follow-up REGEX question in regards to [routeAll] and [routeSubset].
So if I edit the following:
Edit $SPLUNK_HOME/etc/system/local/outputs.conf
[tcpout]
defaultGroup=nothing <----- setting defaultGroup to "nothing" defines that default group is receiving nothing?
As directed with the following stanzas (below):
"everything" (old and new source feeds) goes to the indexer(s)
"subsidiary" goes only to 3rd party TCP receiver....
[tcpout:Everything]
disabled=false
server=x.x.x.x:9997
[tcpout:Subsidiary]
disabled=false
sendCookedData=false
server=x.x.x.x:1234
Is that correct ???
Question:
I am in the staging phase, so I have not had a chance to test a regex for the following:
Please advise how to write the REGEX for the [routeAll] to send all data to indexer(s); and how to write the REGEX for the [routeSubset] to only send uncooked data to the 3rd party TCP receiver.
I need an example if possible.,
For example, REGEX=(SYSTEM|CONFIG|THREAT), how did the author determine this is the correct expression???
[routeAll]
REGEX=(.)<--------- This is where I would specify all data would continue to the indexer(s)?
DEST_KEY=_TCP_ROUTING
FORMAT=Everything
[routeSubset]
REGEX=(SYSTEM|CONFIG|THREAT) <--------- This is where I would specify which data would go to the 3rd party app?
DEST_KEY=_TCP_ROUTING
FORMAT=Subsidiary
Any documentation on this is appreciated.
Thank you
Hi @Log_wrangler,
If you want to send all data (Assuming multiple sourcetypes) are sending data to HF and HF to IDX then you can do following configuration.
outputs.conf
[tcpout]
defaultGroup=Everything
[tcpout:Everything]
disabled=false
server=x.x.x.x:9997
[tcpout:Subsidiary]
disabled=false
sendCookedData=false
server=x.x.x.x:1234
props.conf for only those sourcetype for which you want to transfer data to 3rd party server.
[yoursourcetype]
TRANSFORMS-routesubset = routesubsetdata
transforms.conf, in below configuration if your event contains SYSTEMS OR CONFIG OR THREAT
word then those event will route to 3rd party but those events goes to Indexers as well due to defaultGroup=Everything
. So based on your requirement you can create regex or if you need help then you can ask new question with some sample data for regex.
[routesubsetdata]
REGEX=(SYSTEM|CONFIG|THREAT)
DEST_KEY=_TCP_ROUTING
FORMAT=Subsidiary
I hope this helps.
Thanks,
Harshil
Hi @Log_wrangler,
If you want to send all data (Assuming multiple sourcetypes) are sending data to HF and HF to IDX then you can do following configuration.
outputs.conf
[tcpout]
defaultGroup=Everything
[tcpout:Everything]
disabled=false
server=x.x.x.x:9997
[tcpout:Subsidiary]
disabled=false
sendCookedData=false
server=x.x.x.x:1234
props.conf for only those sourcetype for which you want to transfer data to 3rd party server.
[yoursourcetype]
TRANSFORMS-routesubset = routesubsetdata
transforms.conf, in below configuration if your event contains SYSTEMS OR CONFIG OR THREAT
word then those event will route to 3rd party but those events goes to Indexers as well due to defaultGroup=Everything
. So based on your requirement you can create regex or if you need help then you can ask new question with some sample data for regex.
[routesubsetdata]
REGEX=(SYSTEM|CONFIG|THREAT)
DEST_KEY=_TCP_ROUTING
FORMAT=Subsidiary
I hope this helps.
Thanks,
Harshil
Thank you for the clarification.
What happens if the following is in place?
[tcpout]
defaultGroup=nothing
If you give defaultGroup=nothing
in that case you need to configure props.conf and transforms.conf to route those data to Indexer and 3rd party system as given by you in question. But you need to do same configuration for all sourcetypes otherwise sourcetypes which are not configured to routeall data to Indexers will be dropped on HF.
Thank you, that is what I was concerned about....