I have a requirement to forward search results of a query to an indexer of an external organization. The volume of this data would be fairly high.
I understand there are a multiple ways to achieve this. I am thinking to use a script to run every 5 mins to grab the search results via REST API and store it locally on the disk and forward it from there via outputs.conf
I also understand this would be very to do via script but only challenge is I am not that experienced with scripting stuff, hence little unsure.
Hence, wondering if anyone can please share if there would be an easier way than doing this via a script.
Hi @dm1
Have you tried _TCP_ROUTING in transforms conf of HF.
The search results that you wish to export should have been going through HF in your infra to your internal indexers, if you know exactly what streams you want to forward filter on search pattern/host/source/index etc then send them to external org indexers at the same time using transforms conf _TCP_ROUTING option. No need of scripting.
The search results are not going via HF. This is running a search on already indexed data (its a summary index)
Basically, I am running a search, e.g.
index=abc field1=def field2=ghi
I want to forward the results of the above search to another Indexer on a cron schedule like every 5mins.
@dm1 i mean't the _raw stream at the time of indexing to your internal org indexers going via HF. Same you would like to export after indexing the data and forward it to external org indexers isn't it?