topic Re: forward indexes to different port of the same server in Getting Data In

How to forward indexes to different port of the same server?

Adpafer — Tue, 05 Sep 2023 19:14:52 GMT

Dear Support,

I have 2 indexes (indexA, indexB) and one receiving server with 2 different ports (10.10.10.10:xx, 10.10.10.10:yy). I need my indexer to forward indexA to 10.10.10.10:xx and indexB to 10.10.10.10:yy. What is best way to achieve it? I did two different apps with outputs, props, transforms and it does not work. I tried one app with LB and it does not work either.

Example of outputs.conf:

[tcpout]

defaultGroup = group1, group2

[tcpout:group1]

server = 10.10.10.10:xx

forwardedindex. = ???

[tcpout:group2]

server = 10.10.10.10:yy

forwardedindex. = ???

Is it a good way to do it? How should forwardedindexes config look like ? What about props and transforms?

I would appreciate any help.

thanks pawel

Re: forward indexes to different port of the same server

thahir — Tue, 05 Sep 2023 03:03:55 GMT

you can achieve this by modifying the inputs.conf and output.conf. Can you follow the below steps

your input config should be

[monitor:///path/to/data1]
disabled = false
index = your_index1
sourcetype = your_sourcetype1

[tcpout]
defaultGroup = your_index1_group

[tcpout:your_index1_group]
server = 10.10.10.10:xx

[monitor:///path/to/data2]
disabled = false
index = your_index2
sourcetype = your_sourcetype2

[tcpout:your_index2_group]
server = 10.10.10.10:yy

---------------------

and output.conf is below

[tcpout]
defaultGroup = default-autolb-group

[tcpout-server://localhost:PORT1]
compressed = false

[tcpout-server://localhost:PORT2]
compressed = false

Re: forward indexes to different port of the same server

gcusello — Tue, 05 Sep 2023 06:32:13 GMT

Hi @Adpafer,

let me understand because I don't understand your requirement:

at first if you have one Indexer receiving on port xx and port yy, what do you mean that you need the Indexer forwardrs data on the above ports?

are you speaking of an Indexer or a Forwarder? or are you speaking of forwarding data to a third party?

In other words, could you better describe your requirement, in terms od data flow?

Ciao.

Giuseppe

Re: forward indexes to different port of the same server

Adpafer — Tue, 05 Sep 2023 07:34:51 GMT

HI 🙂

My indexer has to forward some logs to Qradar to 2 different ports:

logs from index A > Qradar port 12468

logs from index B > Qradar port 514

regards, pawel

Re: forward indexes to different port of the same server

SinghK — Tue, 05 Sep 2023 07:42:11 GMT

why are you trying to forward from indexing layer and not from forwarding layer directly. setup the outputs in HF or SF to send data to qradar and splunk instead of from indexers. Ideally I would do that.

Re: forward indexes to different port of the same server

gcusello — Tue, 05 Sep 2023 07:46:22 GMT

Hi @Adpafer,

I suppose that you're sèeaking of forwarding by syslog.

the configurations you used are for sending logs to other Indexers not to a third party

In this case, you should follow the instructions at https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Forwarddatatothird-partysystemsd#Send_a_subset_of_data_to_a_syslog_server

Usually this configuration is used on Heavy Forwarders, not on Indexers, have you HFs in your architecture?

if yes, you can configure them as described in the above documentation,

if not, you could use the Syslog Mod Alert App (https://splunkbase.splunk.com/app/4199), even if isn't certified on Splunk 9.x, but on Search Heads, not on Indexers.

Ciao.

Giuseppe

Re: forward indexes to different port of the same server

Adpafer — Tue, 05 Sep 2023 07:52:31 GMT

It is not my decision. The requirement is to send logs from indexer. I did dedicated app on indexer to send logs from one index to qradar port 514 and it works fine:

outputs.conf:

[tcpout]

forwardedindex.3.blacklist = (.*)
forwardedindex.4.whitelist = (indexA)

[tcpout:tcp_qradar_10_10_10_10_514]

disabled = false
sendCookedData = false
server = 10.10.10.10:514

props.conf:

[source::9997]

TRANSFORMS-routing = send_to_qradar_tcp_10_10_10_10_514

transforms.conf

[send_to_qradar_tcp_10_10_10_10_514]

DEST_KEY = _TCP_ROUTING
FORMAT = tcp_qradar_10_10_10_10_514
REGEX = .

And now I have to add another rule for indexB to be forwarded from indexer to the same IP but port 12468.

I do not how to do it 😞

regards, pawelF

Re: forward indexes to different port of the same server

gcusello — Tue, 05 Sep 2023 08:01:19 GMT

Hi @Adpafer ,

as I said, tcpout is a configuration to send logs from Splunk to another Splunk Indexer, not using syslogs, to use syslogs, you can use the method described in my above url.

What's your architecture: have you a distributed architecture (Searche Heads and Indexers) or a standalone instance?

As I described, the solution depends on it.

probably you need the activity of a Splunk Architect to design your flow.

Ciao.

Giuseppe

Re: forward indexes to different port of the same server

Adpafer — Tue, 05 Sep 2023 09:21:35 GMT

Hi 🙂

I checked this doc https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Forwarddatatothird-partysystemsd?_gl=1*161pvu9*_ga*MTAxNjI5MzU0NC4xNjYyNjM1MTI0*_ga_GS7YF8S63Y*MTY5MzkwNDY1OC40MC4wLjE2OTM5MDQ2NTguMC4wLjA.*_ga_5EPM2P39FV*MTY5MzkwNDY1OC4zNDcuMS4xNjkzOTA0NjYwLjAuMC4w&_ga=2.172375459.1250912549.1693843555-1016293544.1662635124#Send_a_subset_of_data_to_a_syslog_server

and changed from TCP to SYSLOG and it also works:

outputs.conf

[syslog]

forwardedindex.3.blacklist = (.*)
forwardedindex.4.whitelist = (indexA)

[syslog:syslog_qradar_10_10_10_10_514]

disabled = false
sendCookedData = false
server = 10.10.10.10:514

props.conf:

[source::9997]

TRANSFORMS-routing = send_to_qradar_syslog_10_10_10_10_514

transforms.conf

[send_to_qradar_syslog_10_10_10_10_514]

DEST_KEY = _SYSLOG_ROUTING
FORMAT = syslog_qradar_10_10_10_10_514
REGEX = .

And the question is - how to change this config (what should I add) in order to send logs from indexA to 514 and logs from indexB to port 12468 ?

regards, pawelF

Re: forward indexes to different port of the same server

gcusello — Tue, 05 Sep 2023 13:35:07 GMT

Hi @Adpafer,

if you can find a regex to identify one or both the data flows you can create two stanzas in all the configuration files.

If you cannot, you could use the App I hinted before because it uses a search.

Ciao.

Giuseppe