topic Re: Route data to specific indexer based on an external mapping table in Getting Data In

Route data to specific indexer based on an external mapping table

jldgomes — Fri, 17 Jun 2022 08:47:41 GMT

Hello everyone

I'm fairly familiar with routing data based on the logs themselves, however, I was wondering if there was a way to call an external mapping table in the transfoms.conf file.

Logs would contain one identifiable serial number

Firewall 1 with serial number xxxxxxxxxxxx

Firewall 2 with serial number yyyyyyyyyyyy

Firewall 3 with serial number zzzzzzzzzzzz

And we would like to send each log to a different indexer depending on that serial number.

Serial numbers are included in the logs and we have a mapping table that looks like this:

serial number indexer

xxxxxxxxxxxx indexer 1

yyyyyyyyyyyy indexer 2

zzzzzzzzzzzz indexer 3

and so on...

The only way I see right now is to create one manual entry in the props and transform files and I was wondering if there was a way to call an external mapping table, that way, whenever a new firewall comes into play, we would only need to update the table and not props and transforms files.

Thank you

Re: Route data to specific indexer based on an external mapping table

gcusello — Fri, 17 Jun 2022 09:08:43 GMT

Hi @jldgomes,

for my knowledge it isn't possible: you can only create a static association in props.conf and transfoms.conf between one or more regex rules (the serial number) and one or more indexers.

If these different serial numbers aren't too many, this is a way to have an higher control on your ingestions.

Ciao.

Giuseppe

Re: Route data to specific indexer based on an external mapping table

jldgomes — Fri, 17 Jun 2022 09:14:16 GMT

Thanks @gcusello

That was also my assumption. We are talking about 200-ish entries, which is "doable" but it would have been easier if there was another way.