topic Re: inputs.conf TCP One port multiple indexes ? in Getting Data In

inputs.conf TCP One port multiple indexes ?

Silah — Thu, 25 Jul 2024 15:15:27 GMT

Hey all

I am taking input over TCP by having this in my inputs.conf

[tcp://1.2.3.4:123] connection_host = ip index = index1 sourcetype = access_combined

My question is, can I have the same port send data to multiple indexes? Ie. without opening additional ports on my firewall, can I have another host send data to the same port but land in a different index?

I tried adding this

[tcp://5.6.7.8:123] connection_host = ip index = index2 sourcetype = access_combined

but that just stopped the ingestion altogether.

Thanks.

Re: inputs.conf TCP One port multiple indexes ?

gcusello — Thu, 25 Jul 2024 16:22:13 GMT

Hi @Silah ,

yes you can create two different stanzas, one for each sender with different indexes.

The only question is: why?

usually index are choosen when you have different retentions or different access grants, not different sources or technologies.

Different sources are recognized in the same index by host and different technologies are recognized by sourcetype.

Ciao.

Giuseppe

Re: inputs.conf TCP One port multiple indexes ?

Silah — Fri, 26 Jul 2024 07:19:39 GMT

Thanks Guiseppe

The Why: I do need different access grants for one, and I have limitations I am trying to overcome. My heavy forwarders are behind firewall and I have a directive to reduce as far possible the amount of ports open, and ideally I want as little software footprint as possible (so no splunk agents installed on the app servers) so I am trying to use existing syslog forwarder. The TCP forwarding is working fine for the POC but I need to scale it.

Forgive my ignorance reg stanzas, but is that not I tried to do? adding the second [tcp://5.6.7.8:123] ? This didn't work

Re: inputs.conf TCP One port multiple indexes ?

gcusello — Fri, 26 Jul 2024 07:31:44 GMT

Hi @Silah ,

ok, you can use syslog using different stanzas as you did,

if the second one doesn't run, check if the firewall routes are open, you can check this using telnet on the source systems.

In addition I hint to use an rsyslog or a syslog-ng server to take syslog events, instead Splunk TCP inputs, writing them on files and then reading those files with the HF; in this way you can continue to receive logs even if Splunk is down or in maintenance and you'll have less issue for the load of the Splunk Server.

Ciao.

Giuseppe

Re: inputs.conf TCP One port multiple indexes ?

Silah — Fri, 26 Jul 2024 08:09:52 GMT

Thanks, I figured it out using the stanzas.

Don't know if this is the "sanctioned" way but if anyone else are interested, what solved it for me was adding host to each. Without it, it wouldn't work.

So changing this format

[tcp://1.2.3.4:123]
connection_host = ip
index = index1
sourcetype = access_combined

To this:

[tcp://1.2.3.4:123] connection_host = ip host = 1.2.3.4 index = index2 sourcetype = access_combined

[tcp://5.6.7.8:123] connection_host = ip host = 5.6.7.8 index = index2 sourcetype = access_combined