How to load data into multiple indexes based on the event data?

khreddy777 — Thu, 17 Nov 2016 21:45:14 GMT

Can I dynamically assign index value at the indexer level to the events based on the data and load the data into the right index?
For example:
1,A,200
2,A,300
3,B,200
4,B,300
5,C,500
6,C,300

I have created three indexes indexA, indexB, indexC
I want to load the data into indexA,indexB,indexC based on the second column.

After data is loaded:
in IndexA I should have
1,A,200
2,A,300

in indexB I should have
3,B,200
4,B,300

in indexC I should have
5,C,500
6,C,300

Re: How to load data into multiple indexes based on the event data?

neelshah — Tue, 29 Sep 2020 11:50:11 GMT

You may do this by using props.conf and transforms.conf.

-----------------props.conf---------------------

[index_A]
TRANSFORMS-filter_A_events= filter_A_events

[index_B]
TRANSFORMS-filter_B_events= filter_B_events

[index_C]

TRANSFORMS-filter_C_events= filter_C_events

----------Transforms.conf--------------------------

[filter_A_events]
REGEX =
DEST_KEY = queue
FORMAT = indexQueue

[filter_B_events]
REGEX =
DEST_KEY = queue
FORMAT = indexQueue

[filter_C_events]
REGEX =
DEST_KEY = queue
FORMAT = indexQueue

If you are using HF to forward events, place these .conf files on HF.
If you are using UF for forward events, place these .conf files on Indexer

Hope this helps ...!!!

topic Re: How to load data into multiple indexes based on the event data? in Getting Data In

How to load data into multiple indexes based on the event data?

Re: How to load data into multiple indexes based on the event data?

TRANSFORMS-filter_C_events= filter_C_events