How many indexes should I use?

[andrewtrobec]

Hello All,

I have some sizing questions and wanted some input from the community. I'm pretty sure the answer, like most, will be "it depends", but I'm looking for some pointers that I feel are outside of my technical skills. The scenario I'm working with starts with 5 logs. My metric calculations will require me to correlate data from 1 or more logs, even all 5 in some cases. My questions are these:

1. Is it better to create 1 log per index and end up with 5 total indexes, or like the _internal index, should I create a single index for all logs?
2. Are there any specific considerations that would drive this decision? For example, data retention policies are the same for all logs.

Thanks in advance!

Best regards,

Andrew

[gcusello]

Hi andrewtrobec,
the main rules to choose the strategy for indexes are the following:

• retention policy,
• user access to the data.

In few words you have at least to put in the same index logs with the same retention policy and the same grants for user access.

In addition, you can consider also two additional factors:

• put in the same index logs with similar volume of data: in few words, don't put in the same index data from a flow with few logs and data from a flow with many logs;
• logically divide your logs in indexes without exagerate: too many indexes aren't manageable.

Ciao.
Giuseppe