Getting Data In

Shared Index Distributed Search

Communicator

Greetz,

Is it possible to have for example:

  • a search head.
  • multiple search peers with each node having access to the same shared index via SAN.

to speed up concurrent search performance. We have 6+ concurrent users with heavy summary indexing and dashboards with hidden saved searches etc. all on one receiver/indexer.

All I have seen so far is distributed indexes for distributed searches.
Someone asked about cloning indexes but this will only produce duplicate events in results was the answer.
I have since writing this question looked at several posts regarding cloned indexers
and saw using 'dedup' et al.

We were thinking of adding search peers and mounting the shared index via NFSv4 read-only
as our current indexer receives parsed, cooked data from our heavy forwarder and that goes
straight into the index queue anyway. So, maybe a bit less load but still WAY to slow.
So the above host will be the only one writing an index and the rest will merely be reading so no file locking problems?

I was thinking and am probably horribly off the mark here but can you try the above and along with NTP to ensure accurate results:

split the search into three parts based on say, two hour intervals and have that sent to each search node asking A to do the first two hours, B the next two hours and C the last two and so forth?
Can this also be achieved through knowledge objects?
Do cloned indexers require an additional license if they read NFS exports?

Am I right in saying that sending the search to the head and that along to the three nodes as is with all three having the "same" shared index you will get results in triplicate?
But the above will require careful crafting of searches not to mention changing existing ones?

Do each of the search nodes do an equal amount of heavy lifting and present the results back to the head or does the head process the received results and then present to the user?

Thanks.

Super Champion

This question is quite old, but for the benefit of those following along, I think the best answer, as of Splunk 5.x and later, is clustered indexing. See About Clusters in the docs.

0 Karma

Communicator

We cannot use clustering in our environment.

0 Karma

Communicator

We tried exporting an index read-only via NFSv4 and mounted that on the cloned instance.

Splunk refuses to run if $SPLUNK_DB is read-only.

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!