topic Index all and selective forward in Getting Data In

Index all and selective forward

Adevill — Wed, 17 Mar 2021 13:37:00 GMT

Hi all. I need some help to index all data coming into one server and only forward 3 sourcetypes to a 2nd server. Receiving and indexing the data is not a problem, but I cannot seem to get the 3 sourcetypes to the 2nd server. Any help would be appreciated.

My props.conf

[cisco:asa] TRANSFORMS-routing=gsoc [icsp] TRANSFORMS-routing=gsoc [syslog] TRANSFORMS-routing=gsoc

transforms.conf

[gsoc] REGEX=. DEST_KEY=_TCP_ROUTING FORMAT=gsocPrimary

and outputs.conf

[tcpout] defaultGroup=nothing indexAndForward=true [tcpout:gsocPrimary] server=*.*.*.*:9997

Re: Index all and selective forward

Vardhan — Wed, 17 Mar 2021 14:00:18 GMT

Hi @Adevill,

Are you trying to forward the data from HF?

The connectivity between source and destination is exist?

Try the below outputs

[tcpout]
defaultGroup=none
indexAndForward=true

[tcpout:gsocPrimary]
server=*.*.*.*:9997

And why are using the 9997 port why can't use a port like 514?

The 9997 port is already is used to get the data from the forwarder to an indexer. Don't use the same port for two different activities.

Re: Index all and selective forward

Adevill — Wed, 17 Mar 2021 14:09:34 GMT

Hey @Vardhan

Yes, I'm trying to forward from HF to a test server at the moment, that's why the port 9997 doesn't matter now, but you are correct, I would have chosen a different one for deployment. Connectivity is not a problem as I can forward all data to the 2nd server, but it fails when trying to filter for only the 3 sourcetypes. The solution you suggested also didn't work unfortunately.

Re: Index all and selective forward

Vardhan — Wed, 17 Mar 2021 14:15:06 GMT

@Adevill just give a try by keeping seperate stanza's in transform.conf.

props
[cisco:asa]
TRANSFORMS-routing=gsoc1
[icsp]
TRANSFORMS-routing=gsoc2
[syslog]
TRANSFORMS-routing=gsoc3

[gsoc1]
REGEX=(.*)
DEST_KEY=_TCP_ROUTING
FORMAT=gsocPrimary
[gsoc2]
REGEX=(.*)
DEST_KEY=_TCP_ROUTING
FORMAT=gsocPrimary
[gsoc3]
REGEX=(.*)
DEST_KEY=_TCP_ROUTING
FORMAT=gsocPrimary

Re: Index all and selective forward

Adevill — Wed, 17 Mar 2021 14:22:48 GMT

Hi @Vardhan

Unfortunately it's also not working.

Re: Index all and selective forward

Vardhan — Wed, 17 Mar 2021 14:49:32 GMT

@Adevill Can u try with one source type first and check the result

Re: Index all and selective forward

Adevill — Thu, 18 Mar 2021 07:34:22 GMT

Hi @Vardhan

Even if I try just 1 sourcetype it doesn't work.

I've then removed the forwarding, then re-enabled it for all tags, which worked, then changed to a single sourcetype again which failed then again. Any other ideas?