Distribution of Load and Splunk Performance

[nikhilmehra79]

As a quick check can some one suggest me if we have a 2 indexer envirornment with 2 search heads - does it make sense for all universal forwarders to send events to both indexers?

Question i am asking is i am looking to improve performance at my dashboards run from search head and if we are sending same data to both indexers how does it help performance? ( i know it helps from load balancing and DR prospective but not how it helps in seek time/performance)?

Anyone?

[martin_mueller · Accepted solution]

Usually you'll want to have each UF send to both indexers in a load-balanced fashion, so they switch between one and the other. That way almost every search can pull data from both indexers and you get decent performance.
I don't see how cloning the data to both indexers would help performance.

[nikhilmehra79]

Yes both peers are in same cluster controlled by same master node and i have a replication factor 2. Thanks for explanations guys.

[martin_mueller]

If both indexers are peers in the same cluster and your replication factor is 2 then yes, both peers will store each event even if your forwarders only send it to one using load balancing.

[nikhilmehra79]

Ok i think i was not clear with my requirements:

1. I need to send data in Load Blanced method (round Robin) to one indexer at a time

2. But data should synch between two indexers (by peer clustering at port 8080 default). First - am i doing something crazy?, all i am thinking to use efficiently networks and also not give up on a DR.

Will following load balance and send one event to one indexer only on network, and since both indexers are cluster peers will ultimately both indexers have same set of events?
[tcpout:productionSplunk] server = X.X.X.X:9997, Y.Y.Y.Y:9997

[martin_mueller]

@linu1988: That will load-balance rather than clone - note, autoLB=true is the default.

@nikhilmehra79: http://docs.splunk.com/Documentation/Splunk/6.0.2/Admin/outputsconf

If you're talking about one indexer receiving data from the other, that's clustering - are you using clustering?

[nikhilmehra79]

Can you please explain or refer me to doc -

Isn't in both cases UF sent data to one indexer but other indexer recieve it from its first peer at port 8080?

[linu1988]

Martin

[tcpout:productionSplunk]
server = X.X.X.X:9997, Y.Y.Y.Y:9997
will not this clone data as well?

i thought only the below would send load balanced

[tcpout:productionSplunk]
server = X.X.X.X:9997, Y.Y.Y.Y:9997
autoLB=True

[tcpout:productionSplunk]
server = some.splunk.com:9997

load balancer handles the traffic.

[martin_mueller]

They do entirely different things, so which one is okay for you depends on what you want.

[nikhilmehra79]

Thanks so does either of the two configs will be ok?
Option 1
[tcpout]
defaultGroup = productionSplunk

[tcpout:productionSplunk]
server = X.X.X.X:9997, Y.Y.Y.Y:9997

Option 2
autolb=true
[tcpout] defaultGroup = productionSplunk1, productionSplunk2

[tcpout:productionSplunk1] server = X.X.X.X:9997

[tcpout:productionSplunk2] server = Y.Y.Y.Y:9997

[martin_mueller]

Defining two groups would indeed clone data to both indexers. To get load balancing, add two servers to one group like this:

[tcpout]
defaultGroup = productionSplunk

[tcpout:productionSplunk]
server = X.X.X.X:9997, Y.Y.Y.Y:9997

[linu1988]

that would send everything to both, use autolb=true or use a DNS load balancer

[nikhilmehra79]

when you say load balnced way you mean output.conf has ?

[tcpout]
defaultGroup = productionSplunk1, productionSplunk2

[tcpout:productionSplunk1]
server = X.X.X.X:9997

[tcpout:productionSplunk2]
server = Y.Y.Y.Y:9997