topic Re: How to send events from same path to different indexes depending on host using a single deployment? in Getting Data In

How to send events from same path to different indexes depending on host using a single deployment?

mfrost8 — Tue, 15 Jul 2014 17:41:52 GMT

Hello. Here's my situation. I am using the deployment server to push deployments to universal forwarders and would like to create a single deployment for multiple Apache servers. For reasons I won't get into, I have a need to send events from the same path to different indexes depending on the host that they come from.

So the logic of a hypothetical inputs.conf I create would be

[monitor:///var/weblogs/*/*.log]
 if host::host1 OR host::host2 OR host::host3, index = special_index

[monitor:///var/weblogs/*/*.log]
 if host::host4 OR host::host5 OR host::host6, index = main

Obviously inputs.conf doesn't support this kind of syntax, but it's unclear to me how I might be able to accomplish this same thing, if at all, using just one deployment. I already have a lot of different individual deployments with minor tweaks between them like this directing to different indexes stuff, but it's hard to maintain all those different but similar configurations.

Is there a way I might change the index value via configuration for events from this path depending on the host value?

Thanks very much.

Re: How to send events from same path to different indexes depending on host using a single deployment?

martin_mueller — Tue, 15 Jul 2014 18:02:21 GMT

You have two options.

First, create two serverclasses - one for events going to main and one for events going to special_index. That's the easiest to do and most efficient to process for your machines.

Second, you could set up transforms.conf rules on your indexers that decide based on an event's host whether to send an event to main or to special_index. That works, but is a bit harder to configure and adds unnecessary load to your indexers compared to just setting things in inputs.conf right away..

Re: How to send events from same path to different indexes depending on host using a single deployment?

mfrost8 — Tue, 15 Jul 2014 18:20:26 GMT

Thanks, Martin.

That's what I was afraid of. I already have separate deployments for these different hosts which is a pain to maintain because 95% of the deployments are identical so if I make a change I have to make sure I put it in multiple places the same way.

As I was writing the original message, I thought about the indexer-side transforms.conf stuff, but that's not super-clear either. Doesn't seem like there's a great solution for this other than finding a justification for collapsing it all into the same index starting now.

Thanks

Re: How to send events from same path to different indexes depending on host using a single deployment?

martin_mueller — Tue, 15 Jul 2014 19:34:55 GMT

You could put the 95% into a common serverclass and only keep the 5% in separate serverclasses. That should severely reduce maintenance overhead.